Commit graph

963 commits

Author SHA1 Message Date
Chris Lu e5a2bf1287 s3: deprecating filer.options.buckets_fsync 2021-05-22 00:24:23 -07:00
Chris Lu da0a4e775b Revert "Revert "Merge pull request #2027 from bingoohuang/master""
This reverts commit d74cdf0115.
2021-05-10 00:13:55 -07:00
Chris Lu 7ca75347ec minor 2021-05-07 21:56:45 -07:00
Chris Lu 8f8738867f add retry to assign volume
fix https://github.com/chrislusf/seaweedfs/issues/2056
2021-05-07 07:29:26 -07:00
Chris Lu aaad4b578b
Merge pull request #2049 from qieqieplus/async-assign
ahead of time volume assignment
2021-05-06 10:00:25 -07:00
qieqieplus c4d32f6937 ahead of time volume assignment 2021-05-06 18:55:44 +08:00
Chris Lu c899bdf063 a little optimization 2021-05-06 03:03:00 -07:00
Chris Lu 3eb336e0b0 report error only for the first multipart upload
the glog.Errorf would always print for s3 multipart uploads
2021-05-06 01:57:54 -07:00
Chris Lu ac71117ee6 revert PR #1903 avoid http error: superfluous response.WriteHeader 2021-05-05 15:11:39 -07:00
Chris Lu 24efa31e49
Merge pull request #2045 from qieqieplus/fix-vacuum-commit 2021-05-05 07:41:38 -07:00
qieqieplus ac26080bd2 fix concurrent vacuum & delete panic 2021-05-05 17:54:50 +08:00
Chris Lu e24ba2aadc filer: delete specific tags
fix https://github.com/chrislusf/seaweedfs/issues/2041
2021-05-02 21:53:43 -07:00
Chris Lu d74cdf0115 Revert "Merge pull request #2027 from bingoohuang/master"
Need to revert because docker image build failed. The docker apk package only has go 1.15.
2021-04-30 03:36:15 -07:00
Konstantin Lebedev c2269123d3 fix aws style Etag for chunks 2021-04-28 22:28:05 +05:00
bingoohuang 7a9d27fce8 promote to go:embed instead of github.com/rakyll/statik 2021-04-27 17:22:24 +08:00
bingoohuang cf552417a7 minFreeSpace refactored 2021-04-27 10:37:24 +08:00
bingoohuang 31f1cdeac2 minFreeSpace argument allows size like 10GiB 2021-04-26 18:48:34 +08:00
Chris Lu ddc8643ee0 filer: directory listing adds namePatternExclude
fix https://github.com/chrislusf/seaweedfs/issues/2023
2021-04-24 11:49:03 -07:00
Chris Lu f0ad172e80 shell: show which server holds the lock
fix https://github.com/chrislusf/seaweedfs/issues/1983
2021-04-22 23:56:35 -07:00
Chris Lu 11c120c040 master UI adds volume size limit 2021-04-22 14:22:48 -07:00
Chris Lu 6e5df901e4 adjust package names 2021-04-22 14:22:48 -07:00
Konstantin Lebedev 198688c717 revert volume etag 2021-04-16 23:22:31 +05:00
Chris Lu 1e033d45b8 simpler logic
related to https://github.com/chrislusf/seaweedfs/pull/1981
2021-04-12 12:04:53 -07:00
Chris Lu 52200a903b
Merge pull request #1981 from ueni-ltd/if-modified-since
Fix If-Modified-Since behavior
2021-04-12 12:02:54 -07:00
Chris Lu 519b0e1e49 filer: upload to a directory without "/" suffix
fix https://github.com/chrislusf/seaweedfs/issues/1988
2021-04-12 11:56:56 -07:00
Chris Lu af313dff58 add gateway for easier POST and DELETE blobs 2021-04-10 23:47:47 -07:00
Chris Lu 0b82edc0d2 filer: avoid stuck uploader
fix https://github.com/chrislusf/seaweedfs/issues/1980

reverting the file upload batch executor
2021-04-09 13:05:15 -07:00
Merlin Gaillard f952f979d1 filer: return 304 when If-Modified-Since == Last-Modified 2021-04-09 15:04:17 +02:00
Chris Lu 2327c0756b fix to avoid loop 2021-04-05 23:24:26 -07:00
Chris Lu c131764c34 ensure error is sent back in channel
fix https://github.com/chrislusf/seaweedfs/issues/1966

avoid shared readErr variable
2021-04-05 19:40:12 -07:00
Chris Lu 8251d1140e refactor 2021-04-04 18:38:33 -07:00
Chris Lu 8e404a1433 go fmt 2021-04-02 02:22:26 -07:00
Chris Lu 67e019d54b add missing changes 2021-04-02 01:11:44 -07:00
Chris Lu 7e8edc3c4a refactoring 2021-04-02 01:10:24 -07:00
Chris Lu cefe66f159 dedicated upload processor
avoid thundering effect of overloading volume servers
2021-04-01 02:21:40 -07:00
Chris Lu b5880334fc refactor 2021-03-30 21:07:34 -07:00
Chris Lu e79e2ddeed remove unused MoveEvents 2021-03-30 20:42:44 -07:00
Chris Lu ac875976c0 filer, volume: add concurrent upload size limit to avoid OOM
add some back pressure when writes are slow
2021-03-30 02:10:53 -07:00
李海 06be5dc6c3 log snowflake sequencer nodeid's hex when start, it'll be in part of new assigned fid 2021-03-25 18:59:40 +08:00
李海 69b2dab9c6 add a snowflake sequencer as more robust fid generator, but less compressable than small auto-inc id 2021-03-25 18:49:26 +08:00
Chris Lu a801332b0d filer: return 409 if file conflicts with a directory
fix https://github.com/chrislusf/seaweedfs/issues/1938
2021-03-24 01:03:11 -07:00
Chris Lu 5d931eff27 avoid possible nil
fix https://github.com/chrislusf/seaweedfs/issues/1928

The nil was because of `dn.Parent().UnlinkChildNode(dn.Id())` in topo.UnRegisterDataNode() function, when the dn leaves the cluster.
2021-03-22 13:24:07 -07:00
Chris Lu f315eb2bb8 fix build error 2021-03-22 00:12:53 -07:00
Chris Lu b465095db1 shell: add volume.check.disk to fix inconsistency for replicated volumes
fix https://github.com/chrislusf/seaweedfs/issues/1923
2021-03-22 00:03:16 -07:00
Chris Lu b1a86cf808 s3: copy object to itself
fix https://github.com/chrislusf/seaweedfs/issues/1922
2021-03-19 01:31:49 -07:00
Konstantin Lebedev 06da02739d CheckAllChunkViews() for HEAD requests only 2021-03-16 14:15:17 +05:00
Konstantin Lebedev 90510e3137 If WriteHeader is not called explicitly, the first call to Write
will trigger an implicit WriteHeader(http.StatusOK).
WriteHeader: Only one header may be written. Go does not currently!
2021-03-15 21:22:59 +05:00
Konstantin Lebedev 46b9f5cff4 add debug logging 2021-03-15 19:33:33 +05:00
Chris Lu 5d446673a6 fix error printing 2021-03-10 17:13:04 -08:00
Chris Lu df72dc206d stats collect chunk upload retry count 2021-03-07 11:26:15 -08:00