Commit graph

147 commits

Author SHA1 Message Date
Chris Lu ef9c32ea0d adjust compression threshold 2020-10-23 23:05:55 -07:00
Chris Lu 29c9fa2ef2 byte buffer for uploading 2020-10-23 11:29:51 -07:00
Chris Lu 2579edbc60 fix bug found by tests 2020-10-23 10:05:40 -07:00
Chris Lu e0c8507d93 memory pool to reduce memory allocation 2020-10-23 00:41:40 -07:00
Chris Lu c31b254248 mount: shortcut when there is only one chunk 2020-10-20 23:48:29 -07:00
Chris Lu f64252023e Revert "a better byte buffer pool"
This reverts commit 59c6422777.
2020-10-20 23:21:54 -07:00
Chris Lu 59c6422777 a better byte buffer pool 2020-10-20 22:53:58 -07:00
Chris Lu 35f6518c36 weed upload: usePublicUrl was not being used on big files
fix https://github.com/chrislusf/seaweedfs/issues/1492
2020-09-25 02:41:38 -07:00
Chris Lu daf0a449f7 properly cancel context for streaming grpc 2020-09-09 12:07:15 -07:00
Chris Lu 387ab6796f filer: cross cluster synchronization 2020-09-09 11:21:23 -07:00
Chris Lu 50f7e2e89f add log messages 2020-08-26 09:51:20 -07:00
Chris Lu 5f55a87101 close http response 2020-08-24 00:32:44 -07:00
Chris Lu 0b301b0b47 consume all response body in order to release requests 2020-08-23 21:32:29 -07:00
Chris Lu 003d48da21 adjust logs 2020-08-15 19:55:28 -07:00
Chris Lu 83cad3da79 add retry file upload 2020-08-11 20:30:11 -07:00
Chris Lu 20e2ac1add filer: store md5 metadata for files uploaded by filer
fix https://github.com/chrislusf/seaweedfs/issues/1412
2020-08-06 10:04:17 -07:00
Chris Lu 4ecfa9879d volume: report Content-MD5 in response header 2020-08-06 05:22:53 -07:00
Chris Lu 1b3a80dd3d non-fatal error 2020-08-01 13:46:52 -07:00
Chris Lu bbc8ae110f report errors 2020-07-14 11:27:00 -07:00
Chris Lu ddec7b2bb9 go fmt 2020-07-14 07:34:16 -07:00
Chris Lu e4ef3814e7 fix compilation 2020-07-10 10:02:59 -07:00
Chris Lu 2ff37ccdbd testing parsing the upload
The mime type is always the value passed in.

Compress or not depends on the content detection, file name extension, and compression ratio.

If the content is already compressed, need to know the content size.
2020-06-24 11:39:12 -07:00
Chris Lu fe60db404a prepare for zstd 2020-06-23 09:12:02 -07:00
Chris Lu 3080c197e3 rename UnCompressData to DecompressData 2020-06-20 08:16:16 -07:00
Chris Lu 97239ce6f1 rename filechunk is_gzipped to is_compressed 2020-06-20 08:15:49 -07:00
Chris Lu e912fd15e3 renaming 2020-06-19 22:45:27 -07:00
Chris Lu 16fe132a20 detect mime type before replicating to other volume servers 2020-06-19 22:11:36 -07:00
Chris Lu 39e72fb23c go fmt 2020-05-10 03:50:30 -07:00
Chris Lu 78422c2f69
Merge pull request #1298 from song-zhang/master
add fsync option for benchmark
2020-04-30 20:16:42 -07:00
Chris Lu 871efa4fc1 refactoring
some previous chunk etag was using md5, which should be wrong.
2020-04-30 17:20:47 -07:00
zhangsong d819865fe7 add fsync option for benchmark 2020-04-30 23:31:08 +08:00
Chris Lu 9cacaf5eca better detect file mime type 2020-04-14 11:32:31 -07:00
Chris Lu 3f1d79512f only detect when not sure about the types 2020-04-14 11:02:05 -07:00
Chris Lu d2d1597850 volume: detect gzip effectiveness only when content type is empty
revert part of 2286eda575
2020-04-14 10:52:53 -07:00
Chris Lu 8a73276602 refactor a bit 2020-04-08 22:33:36 -07:00
Chris Lu bf270d9e8c filer: able to tail meta data changes 2020-04-05 00:51:16 -07:00
Chris Lu eedd33dda3 refactoring 2020-03-28 13:41:58 -07:00
Chris Lu e1911760a7 refactoring 2020-03-27 00:30:55 -07:00
Chris Lu 2286eda575 sampling whether the data can be gzipped 2020-03-26 23:50:48 -07:00
Chris Lu a5ce359999 refactoring 2020-03-26 22:46:49 -07:00
Chris Lu 6d87a0c5f5 batch delete orphan data by file keys 2020-03-25 02:21:15 -07:00
Chris Lu 6ff9e2835e filer: skip resizing image if width or height larger than original image
fix https://github.com/chrislusf/seaweedfs/issues/1239#issuecomment-602140779
2020-03-21 22:16:00 -07:00
Chris Lu 308688c8d0 filer: add back image resizing capability 2020-03-20 20:31:11 -07:00
Chris Lu f251d03673 refactoring 2020-03-20 15:46:16 -07:00
Chris Lu 11372dd548 refactoring 2020-03-20 15:38:04 -07:00
Chris Lu d848d08944 use public url also for deletions 2020-03-18 11:16:45 -07:00
Chris Lu 6566c8e114 weed upload: add usePublicUrl option
fix https://github.com/chrislusf/seaweedfs/issues/1236
2020-03-18 10:50:53 -07:00
Chris Lu f9b3750ad1 volume: handle repeated writes response 2020-03-15 04:16:00 -07:00
Chris Lu ee635dcc36 refactoring 2020-03-14 14:08:00 -07:00
Chris Lu 18cda6adbb cleaner fix 2020-03-14 13:55:32 -07:00