gitlab-com-seaweedfs/seaweedfs
1
0
Fork 0
mirror of https://github.com/seaweedfs/seaweedfs.git synced 2024-01-19 02:48:24 +00:00
Code Issues Projects Releases Wiki Activity
3806 commits 28 branches 278 tags 135 MiB
Commit graph

2264 commits

Author SHA1 Message Date
李远军 8c318470dd Revert "remove fixJpgOrientation" 
This reverts commit de5ca9b2

minimise the revert changes
 2020-07-10 10:28:28 +08:00
李远军 6608cb5f43 Revert "remove fixJpgOrientation" 
This reverts commit de5ca9b2
 2020-07-10 10:08:36 +08:00
cheng.li01 dad1161c70 fix dn.volumes Iterate when write issue 2020-07-08 19:57:19 +08:00
Chris Lu 54d6b3c30f filer: support running multiple mount to different filers avoiding duplicated cache 
fix https://github.com/chrislusf/seaweedfs/issues/1382
 2020-07-07 23:18:44 -07:00
Chris Lu 4bed1663e2 adjust log level 2020-07-07 23:17:31 -07:00
Chris Lu 482d13468b remove println 2020-07-07 23:17:17 -07:00
Chris Lu a2eb680f34 filer: default filer store directory 
1. set default filer store directory
2. set peers, avoiding empty string counted as 1.
 2020-07-07 23:06:48 -07:00
Chris Lu 3c269da37f fix compilation 2020-07-06 09:46:24 -07:00
Chris Lu d198e8c6d6 filer: support cross filer meta data sync if sharing the same store 2020-07-06 09:24:54 -07:00
Chris Lu 0b8cc9b626 aggregate multiple filer metadata chagne events 2020-07-05 23:05:02 -07:00
Chris Lu 49929e0869 rename 2020-07-05 15:52:36 -07:00
Chris Lu 70d8a3a1d3 add SubscribeLocalMetadata without checking persisted meta logs 2020-07-05 15:50:07 -07:00
Chris Lu 55e40b08fc refactoring 2020-07-05 15:43:06 -07:00
Chris Lu 881e0fde2e rename 2020-07-05 08:55:09 -07:00
Chris Lu 353bea8ddb refactoring 2020-07-03 22:25:35 -07:00
Chris Lu ce3630e7a4 refactoring 2020-07-03 22:16:59 -07:00
Chris Lu 59152334c7 default to 1% 2020-07-03 16:42:53 -07:00
Chris Lu acf4a9ea99 volume: min free space default to 1% 2020-07-03 16:42:02 -07:00
Chris Lu 4d1484628a refactoring 2020-07-03 16:41:30 -07:00
Chris Lu faa5c2e89a refactoring 2020-07-03 16:34:31 -07:00
Chris Lu 37d5b3ba12 replication: pass isFromOtherCluster also to EventNotification 
EventNotification is consistent with message queue and metadata logs.
 2020-07-01 08:06:20 -07:00
Chris Lu 31e23e9783 filer: support active<=>active filer replication 2020-06-30 22:53:57 -07:00
Chris Lu df75b5c98d passing ctx for metadata logging 2020-06-28 14:34:51 -07:00
Chris Lu 12f6b683fc default to 8 volumes 
in case all the volumes are allocated for a collection
 2020-06-28 14:34:21 -07:00
Chris Lu 42a338d7b0 FUSE mount: clean up file handles during renaming 
resolve dir rename when file is still open. Need to clean the file handles as soon as possible.

These can happen out of order:
file rename, then file release
file release, then file rename
 2020-06-28 13:41:00 -07:00
Chris Lu 95a1860d65 remove empty calls for list cache 2020-06-28 10:25:54 -07:00
Chris Lu 6498d0c86b remove list cache 2020-06-28 10:21:52 -07:00
Chris Lu 9033a7d369 removed async option 2020-06-28 10:18:32 -07:00
Chris Lu 1bb8cae65d reverting and working 2020-06-28 10:14:17 -07:00
Chris Lu b813fac4a3 1.84 2020-06-28 07:28:21 -07:00
Chris Lu 20d9cfae86 volume: following md5 base64 encoding 
fix https://github.com/chrislusf/seaweedfs/issues/1374
 2020-06-28 07:25:50 -07:00
Chris Lu f0f884edd5 1.83 2020-06-27 15:46:59 -07:00
Chris Lu c2797cc6de add logs 2020-06-27 15:46:53 -07:00
Chris Lu 62260e526b add logging 2020-06-27 12:54:37 -07:00
Chris Lu bef356ce4c since we already know the chunk size, no need to iterate 2020-06-27 12:51:04 -07:00
Chris Lu a808b3b5df incase the memory data is too small 2020-06-27 11:59:15 -07:00
Chris Lu 3dbd51c3c2 a little bit more efficient 2020-06-26 10:02:37 -07:00
Chris Lu 212b6e7d42 error logging 2020-06-26 10:01:55 -07:00
Chris Lu 48b23f2fdd FUSE mount: prevent concurrent modification 2020-06-26 10:00:48 -07:00
Chris Lu 3cec4b3c49 FUSE mount: fix file id written twice? 
fix https://github.com/chrislusf/seaweedfs/issues/1373
 2020-06-25 20:11:49 -07:00
Chris Lu 212c64241b checking ecx file 2020-06-25 16:36:06 -07:00
Chris Lu a4dc25cd22 volume: generate ec shard first, before generating the ec index file. 
possible fix https://github.com/chrislusf/seaweedfs/issues/1364

the ec index file is used to determine whether the ec files are ready or not
 2020-06-25 11:06:58 -07:00
Chris Lu 9fd7cdadf1 fix 2020-06-25 10:45:34 -07:00
Chris Lu 3b638d3994 add more ec encoding logging 2020-06-25 09:43:38 -07:00
Chris Lu e8e0d629af ensure correct file mode 2020-06-24 23:26:54 -07:00
Chris Lu 32a1f22730 more logs 2020-06-24 22:07:53 -07:00
Chris Lu 5be12eea37 zstd fix 2020-06-24 11:39:12 -07:00
Chris Lu 2ff37ccdbd testing parsing the upload 
The mime type is always the value passed in.

Compress or not depends on the content detection, file name extension, and compression ratio.

If the content is already compressed, need to know the content size.
 2020-06-24 11:39:12 -07:00
Chris Lu c21f4ebfee
Merge pull request #1371 from Kimbsen/content_md5_validation 
Optional md5 validation of uploads
 2020-06-24 11:33:44 -07:00
Kimbsen 4eeab2a379 remove unused imports 2020-06-24 13:37:00 +02:00
Kimbsen ffddecebef do md5 validation AFTER decompression 2020-06-24 13:35:13 +02:00
Chris Lu fe60db404a prepare for zstd 2020-06-23 09:12:02 -07:00
Kimbsen 6b1e93ba0b check request header not part header for content-encoding 2020-06-23 17:04:24 +02:00
Kimbsen 38626cb584 don't forget the last few bytes 2020-06-23 16:53:49 +02:00
Kimbsen 8d1f6cd567 added optional md5 verification of uploaded data 2020-06-23 16:24:11 +02:00
Chris Lu a4e3cffe0a S3: DeleteObject API also support deleting a folder recursively 2020-06-22 10:01:00 -07:00
Chris Lu 98493e0152 volume: add default minFreeSpacePercent 
fix https://github.com/chrislusf/seaweedfs/issues/1366
 2020-06-21 08:44:06 -07:00
Chris Lu e7c36f740a refactoring 2020-06-21 08:38:00 -07:00
Chris Lu 8b4a32e782 adjust help message 2020-06-21 08:34:24 -07:00
Chris Lu b4ec04d464 fix the help message 2020-06-21 08:31:20 -07:00
Chris Lu de5ca9b258 remove fixJpgOrientation 2020-06-20 12:50:40 -07:00
Chris Lu 3080c197e3 rename UnCompressData to DecompressData 2020-06-20 08:16:16 -07:00
Chris Lu 97239ce6f1 rename filechunk is_gzipped to is_compressed 2020-06-20 08:15:49 -07:00
Chris Lu a735fd767c adjust uncompress data 2020-06-20 08:01:00 -07:00
Chris Lu ca3516ac6d adjust protoc 2020-06-20 08:00:25 -07:00
Chris Lu e912fd15e3 renaming 2020-06-19 22:45:27 -07:00
Chris Lu 16fe132a20 detect mime type before replicating to other volume servers 2020-06-19 22:11:36 -07:00
Chris Lu 044841c885 master: always clear previous master meta data directory 2020-06-19 20:42:16 -07:00
Chris Lu 2eeb7551f9 less noisy disk size checking 2020-06-19 15:06:33 -07:00
Chris Lu c48b407053 1.82 2020-06-19 13:47:27 -07:00
Chris Lu b78a2dc59b adjust help message 2020-06-19 12:55:09 -07:00
Chris Lu f7a45d448f FUSE mount: lazy loading meta cache 2020-06-19 09:45:42 -07:00
Chris Lu c0283eee1a more test cases 2020-06-19 09:45:42 -07:00
Chris Lu 55b6efb755 fix checking visited nodes 2020-06-19 09:45:42 -07:00
Chris Lu 0e7c1a300b
Merge pull request #1363 from DXist/fix_utc 
Return last modified in UTC
 2020-06-19 08:49:00 -07:00
Rinat Shigapov 320d70d305 return last modified in UTC 2020-06-19 10:26:40 +03:00
Chris Lu 546709c1de check has visited by the boundary 2020-06-18 22:28:17 -07:00
Chris Lu 0e3b970c0c added bounded tree to track exploration boundary 2020-06-18 21:09:58 -07:00
Chris Lu ae1994cbc1 erasure coding: fix cases where there are no .ecj files 2020-06-18 09:52:35 -07:00
Chris Lu b74eced7a3 mount: a fix to prevent possible repeated calls 
related to https://github.com/bazil/fuse/issues/130
 2020-06-17 13:55:16 -07:00
Chris Lu 7ccc31853c hard limit volume file size 2020-06-16 00:27:48 -07:00
Chris Lu 48880edc2c volume: fix typo in minFreeSpacePercent option 2020-06-14 20:37:05 -07:00
Chris Lu a0464fc267 fix help 
fix https://github.com/chrislusf/seaweedfs/issues/1359
 2020-06-12 22:06:21 -07:00
Chris Lu 0d1d83a5a5 mount: add option to auto create dir to mount 2020-06-12 11:46:58 -07:00
Chris Lu 00f96d86a1 fix typo 2020-06-11 10:53:25 -07:00
Chris Lu 63a1db64c5
Merge pull request #1358 from DXist/s3fixes 
Return NotSuchKey error code for bucket S3 DeleteObject method
 2020-06-11 10:37:19 -07:00
Rinat Shigapov 283b749ff1 keep 204 response 2020-06-11 19:00:34 +03:00
Rinat Shigapov fafc41a27f return xml encoded NotFound status code for s3 delete 2020-06-11 17:53:15 +03:00
Rinat Shigapov 2d2c5dfa39 proxy status code of filer delete response 2020-06-11 15:00:47 +03:00
Chris Lu 628b27ef3b purge old cache implementation 2020-06-11 01:50:00 -07:00
Chris Lu b9365de47b fix volume server pprof 2020-06-10 13:10:10 -07:00
Chris Lu 718052656a
Merge pull request #1355 from Kimbsen/add_http_pprof 
Add --pprof to volume servers to enable http pprof handlers
 2020-06-10 13:05:04 -07:00
Chris Lu 95a9686eaf 1.81 2020-06-10 13:00:42 -07:00
Chris Lu c66f357715 report error correctly 2020-06-10 10:59:25 -07:00
Kimbsen 03e8f976d9 added --pprof to volume servers 2020-06-10 11:52:24 +02:00
Chris Lu c38066be89 also add it for webdav 2020-06-09 18:07:03 -07:00
Chris Lu 4598473251 fix possible EOF problem 2020-06-09 18:04:40 -07:00
Chris Lu bb3e86fbd5 auto create non-existing cache directory 
fix https://github.com/chrislusf/seaweedfs/issues/1350
 2020-06-08 08:49:57 -07:00
Chris Lu bdbe5ddbe9 volume server: add a fix for dead lock 
https://github.com/chrislusf/seaweedfs/issues/1353
 2020-06-08 08:12:59 -07:00
Chris Lu b8e346cc86 1.80 2020-06-05 15:29:25 -07:00