Commit graph

3658 commits

Author SHA1 Message Date
Chris Lu a4dc25cd22 volume: generate ec shard first, before generating the ec index file.
possible fix https://github.com/chrislusf/seaweedfs/issues/1364

the ec index file is used to determine whether the ec files are ready or not
2020-06-25 11:06:58 -07:00
Chris Lu 9fd7cdadf1 fix 2020-06-25 10:45:34 -07:00
Chris Lu 3b638d3994 add more ec encoding logging 2020-06-25 09:43:38 -07:00
Chris Lu e8e0d629af ensure correct file mode 2020-06-24 23:26:54 -07:00
Chris Lu 32a1f22730 more logs 2020-06-24 22:07:53 -07:00
Chris Lu 5be12eea37 zstd fix 2020-06-24 11:39:12 -07:00
Chris Lu 2ff37ccdbd testing parsing the upload
The mime type is always the value passed in.

Compress or not depends on the content detection, file name extension, and compression ratio.

If the content is already compressed, need to know the content size.
2020-06-24 11:39:12 -07:00
Chris Lu c21f4ebfee
Merge pull request #1371 from Kimbsen/content_md5_validation
Optional md5 validation of uploads
2020-06-24 11:33:44 -07:00
Kimbsen 4eeab2a379 remove unused imports 2020-06-24 13:37:00 +02:00
Kimbsen ffddecebef do md5 validation AFTER decompression 2020-06-24 13:35:13 +02:00
Chris Lu fe60db404a prepare for zstd 2020-06-23 09:12:02 -07:00
Kimbsen 6b1e93ba0b check request header not part header for content-encoding 2020-06-23 17:04:24 +02:00
Kimbsen 38626cb584 don't forget the last few bytes 2020-06-23 16:53:49 +02:00
Kimbsen 8d1f6cd567 added optional md5 verification of uploaded data 2020-06-23 16:24:11 +02:00
Chris Lu a4e3cffe0a S3: DeleteObject API also support deleting a folder recursively 2020-06-22 10:01:00 -07:00
Chris Lu a73fd7682a grpc upgrading 2020-06-21 09:24:21 -07:00
Chris Lu 98493e0152 volume: add default minFreeSpacePercent
fix https://github.com/chrislusf/seaweedfs/issues/1366
2020-06-21 08:44:06 -07:00
Chris Lu e7c36f740a refactoring 2020-06-21 08:38:00 -07:00
Chris Lu 8b4a32e782 adjust help message 2020-06-21 08:34:24 -07:00
Chris Lu b4ec04d464 fix the help message 2020-06-21 08:31:20 -07:00
Chris Lu de5ca9b258 remove fixJpgOrientation 2020-06-20 12:50:40 -07:00
Chris Lu 3080c197e3 rename UnCompressData to DecompressData 2020-06-20 08:16:16 -07:00
Chris Lu 97239ce6f1 rename filechunk is_gzipped to is_compressed 2020-06-20 08:15:49 -07:00
Chris Lu a735fd767c adjust uncompress data 2020-06-20 08:01:00 -07:00
Chris Lu ca3516ac6d adjust protoc 2020-06-20 08:00:25 -07:00
Chris Lu e912fd15e3 renaming 2020-06-19 22:45:27 -07:00
Chris Lu 16fe132a20 detect mime type before replicating to other volume servers 2020-06-19 22:11:36 -07:00
Chris Lu 044841c885 master: always clear previous master meta data directory 2020-06-19 20:42:16 -07:00
Chris Lu 2eeb7551f9 less noisy disk size checking 2020-06-19 15:06:33 -07:00
Chris Lu 2fcd7a4c9f clean up 2020-06-19 13:53:54 -07:00
Chris Lu c48b407053 1.82 2020-06-19 13:47:27 -07:00
Chris Lu b78a2dc59b adjust help message 2020-06-19 12:55:09 -07:00
Chris Lu f7a45d448f FUSE mount: lazy loading meta cache 2020-06-19 09:45:42 -07:00
Chris Lu c0283eee1a more test cases 2020-06-19 09:45:42 -07:00
Chris Lu 55b6efb755 fix checking visited nodes 2020-06-19 09:45:42 -07:00
Chris Lu 0e7c1a300b
Merge pull request #1363 from DXist/fix_utc
Return last modified in UTC
2020-06-19 08:49:00 -07:00
Rinat Shigapov 320d70d305 return last modified in UTC 2020-06-19 10:26:40 +03:00
Chris Lu 546709c1de check has visited by the boundary 2020-06-18 22:28:17 -07:00
Chris Lu 0e3b970c0c added bounded tree to track exploration boundary 2020-06-18 21:09:58 -07:00
Chris Lu ae1994cbc1 erasure coding: fix cases where there are no .ecj files 2020-06-18 09:52:35 -07:00
Chris Lu ce79ec0bd4 HCFS: 1.2.9 fix concurrent modification 2020-06-18 07:52:32 -07:00
Chris Lu 222f93e816 possibly fix concurrent access to entry object in SeaweedOutputStream 2020-06-17 14:59:43 -07:00
Chris Lu b74eced7a3 mount: a fix to prevent possible repeated calls
related to https://github.com/bazil/fuse/issues/130
2020-06-17 13:55:16 -07:00
Chris Lu a80bf29d4f
Merge pull request #1360 from ingardm/master
modified see_dat to analyze needle compression
2020-06-17 09:55:34 -07:00
ingardm f711e8dde4
Rename see_dat_gzip to see_dat_gzip.go 2020-06-17 10:43:51 +02:00
ingardm b7afa9fd57
Create see_dat_gzip 2020-06-17 10:43:28 +02:00
Chris Lu 0ca4a6c4e6
Update README.md 2020-06-16 21:58:17 -07:00
Chris Lu 7ccc31853c hard limit volume file size 2020-06-16 00:27:48 -07:00
Chris Lu 48880edc2c volume: fix typo in minFreeSpacePercent option 2020-06-14 20:37:05 -07:00
Chris Lu a0464fc267 fix help
fix https://github.com/chrislusf/seaweedfs/issues/1359
2020-06-12 22:06:21 -07:00