gitlab-com-seaweedfs/seaweedfs
1
0
Fork 0
mirror of https://github.com/seaweedfs/seaweedfs.git synced 2024-01-19 02:48:24 +00:00
Code Issues Projects Releases Wiki Activity
4362 commits 28 branches 278 tags 135 MiB
Commit graph

516 commits

Author SHA1 Message Date
Chris Lu c21f4ebfee
Merge pull request #1371 from Kimbsen/content_md5_validation 
Optional md5 validation of uploads
 2020-06-24 11:33:44 -07:00
Kimbsen 4eeab2a379 remove unused imports 2020-06-24 13:37:00 +02:00
Kimbsen ffddecebef do md5 validation AFTER decompression 2020-06-24 13:35:13 +02:00
Chris Lu fe60db404a prepare for zstd 2020-06-23 09:12:02 -07:00
Kimbsen 6b1e93ba0b check request header not part header for content-encoding 2020-06-23 17:04:24 +02:00
Kimbsen 38626cb584 don't forget the last few bytes 2020-06-23 16:53:49 +02:00
Kimbsen 8d1f6cd567 added optional md5 verification of uploaded data 2020-06-23 16:24:11 +02:00
Chris Lu de5ca9b258 remove fixJpgOrientation 2020-06-20 12:50:40 -07:00
Chris Lu 3080c197e3 rename UnCompressData to DecompressData 2020-06-20 08:16:16 -07:00
Chris Lu e912fd15e3 renaming 2020-06-19 22:45:27 -07:00
Chris Lu 16fe132a20 detect mime type before replicating to other volume servers 2020-06-19 22:11:36 -07:00
Chris Lu ae1994cbc1 erasure coding: fix cases where there are no .ecj files 2020-06-18 09:52:35 -07:00
Chris Lu 7ccc31853c hard limit volume file size 2020-06-16 00:27:48 -07:00
Chris Lu bdbe5ddbe9 volume server: add a fix for dead lock 
https://github.com/chrislusf/seaweedfs/issues/1353
 2020-06-08 08:12:59 -07:00
Chris Lu 45a9d852d3 go fmt 2020-06-05 15:27:10 -07:00
Evgenii Kozlov 2410ffbd79 Name fix 2020-06-05 19:00:38 +03:00
Evgenii Kozlov 0e0db70f55 Set volumes ReadOnly if low free disk space 2020-06-05 18:18:15 +03:00
Chris Lu ef2b3a0801 leveldb recover error handling 2020-05-26 00:03:44 -07:00
Chris Lu e5a0787653 filer: leveldb store add auto recovery, report correct error 
fix https://github.com/chrislusf/seaweedfs/issues/1326
 2020-05-22 10:54:42 -07:00
Chris Lu 7ff2b0121d specify constant data type to uint64 instead of default int 
fix https://github.com/chrislusf/seaweedfs/issues/1324
 2020-05-19 07:55:23 -07:00
Chris Lu 11b33c0760 fix compilation problem 2020-05-18 10:53:31 -07:00
zhangsong b161baa3f0 fix volume vacuum test unit 2020-05-07 19:58:06 +08:00
zhangsong 3703c7e176 write request keep compatible with before 2020-05-06 21:35:32 +08:00
zhangsong f9e8702bb4 use async write to persistent file to disk - part1 2020-05-04 17:39:44 +08:00
Chris Lu 6190fd665d printout error 2020-04-27 12:41:31 -07:00
Chris Lu bafa95045b volume: deletion checks all disk locations 
fix https://github.com/chrislusf/seaweedfs/issues/1283
 2020-04-21 14:49:58 -07:00
Chris Lu 7764e0465c refactoring 2020-04-12 21:00:55 -07:00
Chris Lu e4af63a721 volume server: accept fsync=true in write requests 2020-04-11 21:39:16 -07:00
Chris Lu c8ca234773 refactoring 2020-04-11 14:27:25 -07:00
Chris Lu df9d538044 rename function 2020-04-11 14:19:44 -07:00
Chris Lu 9fa065f600 typo in logs 2020-04-09 23:43:09 -07:00
Chris Lu e63a79ade8 better handle lock in case of exception 2020-03-24 18:41:25 -07:00
Chris Lu 3137777d83 volume: automatically detect max volume count 2020-03-22 16:21:42 -07:00
Chris Lu cbfe31a9a8 idx file sync before compaction 2020-03-20 23:38:46 -07:00
Chris Lu 81797a059a volume: sync volume file right before compaction 
fix https://github.com/chrislusf/seaweedfs/issues/1237
 2020-03-19 23:54:52 -07:00
Chris Lu 709f231e23 tiered storage: add s3 endpoint for private s3 implementation 
fix https://github.com/chrislusf/seaweedfs/issues/1238
 2020-03-19 21:13:56 -07:00
Chris Lu b964bbab3d fix compilation 2020-03-17 10:01:24 -07:00
Chris Lu c3cb6fa1d7 volume: compaction can cause readonly volumes 
address https://github.com/chrislusf/seaweedfs/issues/1233
 2020-03-17 09:43:57 -07:00
Chris Lu 22400c6633 consistent gzip logic 
local store gzip same as replicated writes
 2020-03-15 04:15:40 -07:00
Chris Lu 560df51def refactoring 2020-03-15 03:11:26 -07:00
Chris Lu d439d83772 volume: follow compactionBytePerSecond 
related to https://github.com/chrislusf/seaweedfs/issues/1108
 2020-03-11 10:32:17 -07:00
Chris Lu d3e1ef64b3 ensure error checking on windows 2020-03-10 13:33:26 -07:00
Chris Lu 88a110e67e release file handle 2020-03-10 13:32:22 -07:00
Chris Lu 43e6261579 volume: fix memory leak 2020-03-10 13:31:23 -07:00
Chris Lu 0871d2cff0 volume: fix memory leak during compaction 
fix https://github.com/chrislusf/seaweedfs/issues/1222
 2020-03-09 22:29:02 -07:00
Chris Lu 89eb05b50f filer: support TTL for all filer stores 2020-03-09 01:02:01 -07:00
Chris Lu 9b3109a5d8 filer: processing all response headers, no pass through to volume server 
* filer calculate MD5 etag
* filer handle response headers, instread of pass it to volume servers
 2020-03-08 15:42:44 -07:00
Chris Lu 1ae83c2938 go fmt 2020-03-07 06:12:57 -08:00
Chris Lu ea1169dc80 filer cipher: single chunk http POST and PUT and read 2020-03-07 06:08:08 -08:00
Chris Lu 7335e62199 volume: PUT also conditionally gzip compress 2020-03-01 23:37:02 -08:00
Chris Lu 151114ff59 volume: fix readonly status reporting 2020-03-01 17:57:37 -08:00
Chris Lu f87f2045b3 delete old file first on windows 
fix https://github.com/chrislusf/seaweedfs/issues/1210
 2020-02-27 00:59:35 -08:00
Chris Lu 757c7d67ad avoid nil DataBackend 2020-02-27 00:07:25 -08:00
Chris Lu 430b5a49e2 unused code 2020-02-26 19:35:00 -08:00
Chris Lu ed0acd1722 go fmt 2020-02-26 16:52:57 -08:00
Chris Lu 97ab8a1976 remove ctx if possible 2020-02-25 22:23:59 -08:00
Chris Lu 892e726eb9 avoid reusing context object 
fix https://github.com/chrislusf/seaweedfs/issues/1182
 2020-02-25 21:50:12 -08:00
panyc16 7b3764fd9e make loadExistingVolume() return bool 2020-02-13 15:19:00 +08:00
Chris Lu 8d94564f41 refactor 2020-02-04 21:16:34 -08:00
Chris Lu c1288e9eb4 volume: sdx generation uses memdb instead of compactMap 
fix https://github.com/chrislusf/seaweedfs/issues/1194
 2020-02-04 21:12:09 -08:00
Chris Lu 08e4702542 sync before closing index file 
merge from 0181f87d91
 2020-02-04 10:38:12 -08:00
Chris Lu 3a35632d58 sync before closing file 
merge changes from 1747fc2d52
 2020-02-04 10:37:14 -08:00
Chris Lu 40ae533fa3 shell: add volume.configure.replication to change replication for a volume 
fix https://github.com/chrislusf/seaweedfs/issues/1192
 2020-02-02 15:37:23 -08:00
Chris Lu d335f04de6 support env variables to overwrite toml file 2020-01-29 09:09:55 -08:00
Chris Lu 72a64a5cf8 use the same context object in order to retry 2020-01-26 14:42:11 -08:00
Chris Lu b6e6ca8595 adjust logs 2020-01-24 20:06:58 -08:00
Chris Lu c8b2dac6c1 volume: avoid sharing volume dat file handle 
possibly help on https://github.com/chrislusf/seaweedfs/issues/1184
 2020-01-21 21:18:01 -08:00
Chris Lu f8a20ef35e add the old way to compact as a comment 2020-01-08 09:45:42 -08:00
Chris Lu 288baf37fd saving .vif files correctly 2020-01-08 09:45:26 -08:00
Chris Lu acf7ca7b93 volume: fix compaction 2020-01-08 09:45:03 -08:00
Chris Lu 943f4986ef fix possible nil 2020-01-08 08:49:18 -08:00
Chris Lu 9995d3bcb5 remove println 2020-01-07 21:49:28 -08:00
Chris Lu 3eafec4b29 volume: add option to limit file size 2020-01-03 00:37:26 -08:00
Chris Lu 672868b460 always create .vif file 2019-12-28 21:52:06 -08:00
Chris Lu b6f7033478 go fmt 2019-12-28 21:37:29 -08:00
Chris Lu 9134092a8d endure creating .vif files 2019-12-28 21:36:15 -08:00
Chris Lu 0cc8347a3a go fmt 2019-12-28 21:13:10 -08:00
Chris Lu 37b64a50b4 ec: generate and copy .vif file 2019-12-28 12:44:59 -08:00
Chris Lu c06f7eb48a load volume info from .vif file, use superblock as a backup 2019-12-28 12:28:58 -08:00
Chris Lu f4a74e03d1 refactoring: separating .vif from tier file loading 2019-12-28 11:35:27 -08:00
Chris Lu 2000284435 rename volume tier info to volume info 2019-12-28 11:21:49 -08:00
Chris Lu e8b357fd53 rename from .tier to .vif 2019-12-28 11:17:39 -08:00
Chris Lu 48d28d3eb2 tier: support remote file attributes and remember the file extension 2019-12-25 21:37:24 -08:00
Chris Lu 1346437d71
Merge pull request #1171 from stlpmo-jn/redundant_type_conversion 
remove the redundant type conversion
 2019-12-25 17:02:05 -08:00
Chris Lu 35393b4a02 cleanly destroy remote files 2019-12-25 16:17:58 -08:00
Chris Lu eecad6062a go fmt 2019-12-25 09:56:42 -08:00
Chris Lu d960b3474a tier storage: support downloading the remote dat files 2019-12-25 09:53:13 -08:00
stlpmo f3de4b6c18 remove the redundant type conversion 2019-12-25 10:13:45 +08:00
Chris Lu 9ff72f616a go fmt 2019-12-24 14:56:16 -08:00
Chris Lu efd2f50ede compaction changed to .idx based deletion 2019-12-24 14:55:50 -08:00
Chris Lu d8b39fe92a testing 2019-12-24 11:29:26 -08:00
Chris Lu f42b5bd0f5 rename .sdb to .sdx 2019-12-24 10:19:12 -08:00
Chris Lu abffe857a1 change btree map to in memory level db 2019-12-24 10:18:56 -08:00
Chris Lu 72a561ab7c refactoring 2019-12-24 08:13:01 -08:00
Chris Lu 409a3fe41f
Merge pull request #1168 from stlpmo-jn/optimize_DeleteCollectionFromDiskLocation 
decouple the volume.Destroy() from the operation of unmountVolume()
 2019-12-24 07:08:04 -08:00
stlpmo 38e4b79125 decouple the volume.Destroy() from the operation of unmountVolume() 2019-12-24 17:20:34 +08:00
stlpmo f300c35266 resolve issue : when the host disk full, the volume server will core dump 2019-12-24 16:36:15 +08:00
Chris Lu a18f62bbe7 only copy required shards 2019-12-23 18:06:13 -08:00
Chris Lu 8a2d529759 go fmt 2019-12-23 12:48:53 -08:00
Chris Lu 09ca936c78 shell: add ec.decode command 2019-12-23 12:48:20 -08:00