gitlab-com-seaweedfs/seaweedfs
1
0
Fork 0
mirror of https://github.com/seaweedfs/seaweedfs.git synced 2024-01-19 02:48:24 +00:00
Code Issues Projects Releases Wiki Activity
5748 commits 28 branches 278 tags 135 MiB
Commit graph

3688 commits

Author SHA1 Message Date
Chris Lu 55d547afec switch back to in memory uploading for now 2021-05-10 13:50:10 -07:00
Chris Lu 9a6aa00e9d avoid nil locations 
fix https://github.com/chrislusf/seaweedfs/issues/2059
 2021-05-10 02:39:52 -07:00
Chris Lu da0a4e775b Revert "Revert "Merge pull request #2027 from bingoohuang/master"" 
This reverts commit d74cdf0115.
 2021-05-10 00:13:55 -07:00
Chris Lu aa6949ef25 2.44 2021-05-09 23:28:48 -07:00
Chris Lu ba77833343 EOF handling 2021-05-09 23:04:24 -07:00
Chris Lu f4cf30b497 mount: write to tmp file before uploading 2021-05-09 22:56:10 -07:00
Chris Lu 280e7cf289 mount: in case the set attribute is called before persisting the file 2021-05-09 22:55:30 -07:00
Chris Lu 74052064b6 add one missing optimization 2021-05-09 17:25:42 -07:00
Chris Lu 59ace54925 refactor 2021-05-09 17:22:30 -07:00
Chris Lu 50be19d23e refactor 2021-05-09 15:33:01 -07:00
Chris Lu 55e060cf61 refactor 2021-05-09 15:28:54 -07:00
Chris Lu 735e65be17 refactor 2021-05-09 15:22:38 -07:00
Chris Lu 93e84a12f2 refactor 2021-05-09 15:15:18 -07:00
Chris Lu 3942e3b2ef a better fix 2021-05-09 01:42:19 -07:00
Chris Lu 957e1a1bc1 fuse: important: if filer -filer.path is not root, directory listing will fail 2021-05-09 00:32:21 -07:00
Chris Lu 7ca75347ec minor 2021-05-07 21:56:45 -07:00
Chris Lu 8f8738867f add retry to assign volume 
fix https://github.com/chrislusf/seaweedfs/issues/2056
 2021-05-07 07:29:26 -07:00
Chris Lu 007401f3a0 remove duplicated code 2021-05-07 07:14:24 -07:00
Chris Lu aaad4b578b
Merge pull request #2049 from qieqieplus/async-assign 
ahead of time volume assignment
 2021-05-06 10:00:25 -07:00
qieqieplus c4d32f6937 ahead of time volume assignment 2021-05-06 18:55:44 +08:00
Chris Lu 55a8f57381 go fmt 2021-05-06 03:37:51 -07:00
Chris Lu 5753749c90 remove verbose logs 2021-05-06 03:34:34 -07:00
Chris Lu 38f411219a mount: skip local chunk cache if opened write only 2021-05-06 03:31:40 -07:00
Chris Lu c899bdf063 a little optimization 2021-05-06 03:03:00 -07:00
Chris Lu 3eb336e0b0 report error only for the first multipart upload 
the glog.Errorf would always print for s3 multipart uploads
 2021-05-06 01:57:54 -07:00
Chris Lu 76c48ffe27 optional parallel copy ec shards 
fix https://github.com/chrislusf/seaweedfs/issues/2048
 2021-05-06 01:53:35 -07:00
Chris Lu ac71117ee6 revert PR #1903 avoid http error: superfluous response.WriteHeader 2021-05-05 15:11:39 -07:00
Chris Lu 24efa31e49
Merge pull request #2045 from qieqieplus/fix-vacuum-commit 2021-05-05 07:41:38 -07:00
qieqieplus ac26080bd2 fix concurrent vacuum & delete panic 2021-05-05 17:54:50 +08:00
Chris Lu e24ba2aadc filer: delete specific tags 
fix https://github.com/chrislusf/seaweedfs/issues/2041
 2021-05-02 21:53:43 -07:00
Chris Lu e87e6ef33c s3: return 404 if bucket does not exist 
fix https://github.com/chrislusf/seaweedfs/issues/2039
 2021-05-02 21:30:37 -07:00
Chris Lu a4bb37a5fe add debug info 2021-05-01 16:10:26 -07:00
Chris Lu 2e56407c6b fix visited checking 2021-05-01 16:09:29 -07:00
Chris Lu c48ef78670 2.43 2021-05-01 00:39:04 -07:00
Chris Lu 3a86d4dbfd mount: fix directory invalidation 
fix https://github.com/chrislusf/seaweedfs/issues/2038
 2021-04-30 22:51:06 -07:00
Chris Lu d74cdf0115 Revert "Merge pull request #2027 from bingoohuang/master" 
Need to revert because docker image build failed. The docker apk package only has go 1.15.
 2021-04-30 03:36:15 -07:00
Chris Lu 84312e6799 2.42 2021-04-30 03:14:07 -07:00
Chris Lu fa0d973113
Merge pull request #2037 from utsl42/reader_at 
make reader_at handle random reads more efficiently for FUSE
 2021-04-28 17:31:46 -07:00
Nathan Hawkins 042de9359c make reader_at handle random reads more efficiently for FUSE 2021-04-28 19:13:37 -04:00
Chris Lu a26a37dfa3 fix compilation 
fix related to #2032
 2021-04-28 13:36:53 -07:00
Chris Lu ef94ff6837
Merge pull request #2035 from kmlebedev/fix_chunks_etag 
fix aws style Etag for chunks
 2021-04-28 10:37:18 -07:00
Chris Lu 8ae3ea4b1d
Merge pull request #2034 from kmlebedev/iam_listkeys_filter 
iam ListAccessKeys filtred by username
 2021-04-28 10:34:40 -07:00
Chris Lu 9dca75aea8
Merge pull request #2032 from tobiasmuehl/patch-2 
Detect rar archives by mime type
 2021-04-28 10:31:19 -07:00
Konstantin Lebedev c2269123d3 fix aws style Etag for chunks 2021-04-28 22:28:05 +05:00
Konstantin Lebedev 39f636c682 iam ListAccessKeys filtred by username 2021-04-28 17:15:22 +05:00
Tobias Mühl a8864e2abd
Detect rar archives by mime type 
RAR archives might not have .rar extension, see [Wikipedia](https://en.wikipedia.org/wiki/RAR_(file_format))
 2021-04-28 15:54:19 +07:00
Tobias Mühl 12a7e87007
Do not compress brotli archives 2021-04-28 15:51:49 +07:00
Konstantin Lebedev a48785c7df auth use bucket wild cards 2021-04-27 21:45:40 +05:00
bingoohuang 7a9d27fce8 promote to go:embed instead of github.com/rakyll/statik 2021-04-27 17:22:24 +08:00
bingoohuang cf552417a7 minFreeSpace refactored 2021-04-27 10:37:24 +08:00
bingoohuang 31f1cdeac2 minFreeSpace argument allows size like 10GiB 2021-04-26 18:48:34 +08:00
Chris Lu 86185262bb 2.41 2021-04-24 16:54:36 -07:00
Chris Lu 79f2e780c1 ensure name pattern checking is case sensitive 2021-04-24 11:51:23 -07:00
Chris Lu ddc8643ee0 filer: directory listing adds namePatternExclude 
fix https://github.com/chrislusf/seaweedfs/issues/2023
 2021-04-24 11:49:03 -07:00
Chris Lu f0ad172e80 shell: show which server holds the lock 
fix https://github.com/chrislusf/seaweedfs/issues/1983
 2021-04-22 23:56:35 -07:00
Chris Lu 89eb9f6e70 clean up .uploads directory in mysql and postgres tables 
fix https://github.com/chrislusf/seaweedfs/issues/1957

When no uploads are running, you can run this SQL to clean up.
delete from <bucket_name> where directory like '/.uploads/%'
 2021-04-22 23:23:23 -07:00
Chris Lu 46ef1811a1 correct help message 2021-04-22 22:26:38 -07:00
Chris Lu 11c120c040 master UI adds volume size limit 2021-04-22 14:22:48 -07:00
Chris Lu 6e5df901e4 adjust package names 2021-04-22 14:22:48 -07:00
Chris Lu ca998328c2 do not add new volumes when below minFreeSpacePercent 
fix https://github.com/chrislusf/seaweedfs/issues/2017
 2021-04-21 23:11:11 -07:00
Chris Lu cd7bf1a72a filer.copy copy empty folders 
fix https://github.com/chrislusf/seaweedfs/issues/2016
 2021-04-21 11:17:43 -07:00
Chris Lu a8114da02d avoid thundering herd effect 
transient errors may cause thundering herd effect to all trying to recover from remove ec shards
 2021-04-21 10:17:12 -07:00
Chris Lu ae74d8f02a fix error message 
related to https://github.com/chrislusf/seaweedfs/issues/2012
 2021-04-21 01:40:16 -07:00
Chris Lu 11c405fc85 ensure file handles are released 2021-04-20 19:56:55 -07:00
liuxiaobo c31c5e829c
fix path-specific filer store comment error 2021-04-20 10:08:58 +08:00
Chris Lu 83cf94ad2d delay new file creation unless file is opened exclusively 2021-04-19 10:58:25 -07:00
Chris Lu e983f91b03 2.40 2021-04-18 13:58:01 -07:00
Chris Lu d1c813c470 let the fuse library manage directory id 
otherwise, on mac, during large directory deletion, if some ReaDirAll happens, the lib seems confused about the directories, and some child directories are not deleted.
 2021-04-18 13:07:28 -07:00
Chris Lu 372872ebbf set root node inode number 2021-04-18 13:07:28 -07:00
Chris Lu e332da4837 set inode value 2021-04-18 13:07:28 -07:00
Chris Lu 2acf6be24e resend the http request if connection is stale 2021-04-18 13:07:28 -07:00
Chris Lu 6cbd786db9 correctly runs git clone 2021-04-18 13:07:28 -07:00
Chris Lu d41e6826d3 adjust logging 2021-04-18 13:06:38 -07:00
Chris Lu d9a2a7f1c4 WIP 
no memory issue

if some directory is removed, it may have this error

$ rm -Rf ~/tmp/m2/s1
rm: fts_read: Device not configured
 2021-04-18 13:06:38 -07:00
Chris Lu 54410ca955 cleaner way to set readonly 2021-04-18 10:02:02 -07:00
Konstantin Lebedev 198688c717 revert volume etag 2021-04-16 23:22:31 +05:00
Chris Lu c83ab91e2e remove unused variable 2021-04-16 10:34:02 -07:00
Chris Lu 3074e9b428 ensure consistent inode value 2021-04-15 22:42:24 -07:00
Chris Lu b971317a16 avoid possible corrupted file names 2021-04-15 11:41:34 -07:00
Chris Lu 283d703d50 adjust text 2021-04-15 11:29:58 -07:00
Chris Lu 609e228578 avoid forward slash in file names 2021-04-15 10:53:04 -07:00
Chris Lu ba92f2e714 add node.selectedVolumes 
fix https://github.com/chrislusf/seaweedfs/issues/1990
 2021-04-15 03:19:28 -07:00
Chris Lu 16c0304416 ensure to delete on filer also 2021-04-15 02:29:04 -07:00
Chris Lu 217e0f9066 mount: remove folder recursively 2021-04-15 01:51:10 -07:00
Chris Lu 3e669e6d7b mostly refactoring, add some error handling 2021-04-14 23:33:37 -07:00
Chris Lu 07f712c83f fix typo 2021-04-14 23:21:38 -07:00
Chris Lu 36c79de3f4 fuse mount: dir ReadDirAll avoid extra conversion to filer_pb.Entry 2021-04-14 23:21:24 -07:00
Chris Lu e41766feb6 fuse mount: dir lookup avoids extra conversion to filer_pb.Entry object 2021-04-14 22:38:34 -07:00
Chris Lu 1adc8f86ea lighten up File object 
file.entry only exists when file.isOpen
 2021-04-14 20:49:15 -07:00
Chris Lu 6bc09b18c4 truncate is a bit faster to reuse the storage 2021-04-14 20:26:56 -07:00
Chris Lu c04b7e106f mount: remove entry from Dir object 2021-04-14 20:26:13 -07:00
Chris Lu fc0cbf565f add option to obfuscate the file names 2021-04-14 15:37:24 -07:00
Chris Lu e75633c64f volume.check.disk: break loop for read only volumes 
fix https://github.com/chrislusf/seaweedfs/issues/2002
 2021-04-14 12:40:13 -07:00
Chris Lu 9d50867d08 volume.tier.move: avoid data loss when destination volume server already has the volume 
fix https://github.com/chrislusf/seaweedfs/issues/2001
 2021-04-14 10:26:26 -07:00
Chris Lu ff4c1d5965 adjust logging 
fix https://github.com/chrislusf/seaweedfs/issues/1999
 2021-04-14 10:04:26 -07:00
qieqieplus 270645f8d7 fix #1996 2021-04-14 18:29:28 +08:00
Chris Lu 90677e1097 ensure to call line.Close() 
fix https://github.com/chrislusf/seaweedfs/issues/1995

similar to https://github.com/peterh/liner/issues/104
 2021-04-14 01:33:21 -07:00
Chris Lu 3f3268cd1b go fmt 2021-04-14 00:30:16 -07:00
Chris Lu ca0f07a188 move file reader, entryViewCache to file handle 
reduce file object size
 2021-04-14 00:29:58 -07:00
Chris Lu 5985a7d38d add log level possible values 
fix https://github.com/chrislusf/seaweedfs/issues/1989
 2021-04-12 21:15:51 -07:00