gitlab-com-seaweedfs/seaweedfs
1
0
Fork 0
mirror of https://github.com/seaweedfs/seaweedfs.git synced 2024-01-19 02:48:24 +00:00
Code Issues Projects Releases Wiki Activity
6283 commits 28 branches 278 tags 135 MiB
Commit graph

4095 commits

Author SHA1 Message Date
Chris Lu 5adfdd7982 change to a new bloom filter library 2021-08-29 21:26:38 -07:00
Chris Lu 05f32376eb add cluster id in filer configuration response 2021-08-29 21:02:10 -07:00
Chris Lu eacaa44dc2 refactor 2021-08-29 19:13:48 -07:00
Chris Lu 6deee4c0b9 import hdfs 2021-08-29 18:49:43 -07:00
Chris Lu a31f2907f0 cloud drive: filer.remote.sync supports remove folder 2021-08-29 18:46:28 -07:00
Chris Lu 001a472057 cloud mount: remote storage support hdfs 2021-08-29 18:41:29 -07:00
Chris Lu 05d2774117 refactoring 2021-08-28 22:49:38 -07:00
Chris Lu 975fbc9ee3 add wasabi example 2021-08-28 22:49:38 -07:00
Chris Lu bec3f63298 2.65 2021-08-28 05:27:33 -07:00
Chris Lu 223c1b3571 adjust invalidation logic 2021-08-28 05:21:01 -07:00
nivekuil 8c523d50ad Invalidate mount meta cache more aggressively 2021-08-28 04:26:37 -07:00
Chris Lu 65a29ad42a add warning on unmount a folder 2021-08-28 02:25:48 -07:00
Chris Lu a22f37b01c remove tencent region 2021-08-28 02:23:03 -07:00
Chris Lu a9b41900d8 handle cases when mounted directory does not exist 2021-08-28 01:56:56 -07:00
Chris Lu 0d8936f6cd add aliyun example 2021-08-28 01:09:48 -07:00
Chris Lu dc481c081c remove unused function 2021-08-27 22:18:54 -07:00
Chris Lu 9242f3aaeb fix build 2021-08-26 17:33:57 -07:00
Chris Lu 49a8dfb976 adjust default concurrent level 2021-08-26 17:05:56 -07:00
Chris Lu 6a0bb7106b cloud drive: parallelize remote storage downloading 2021-08-26 16:16:26 -07:00
Chris Lu 05a648bb96 refactor: separating out remote.proto 2021-08-26 15:18:34 -07:00
yulai.li c1dc5ab4ac Add deleterange_concurrency to filer configuration file 2021-08-26 18:25:08 +08:00
yulai.li 318757ef8c Change DeleteFolderChildren to DeleteRange api 2021-08-26 17:49:56 +08:00
yulai.li de8ef28460 Update filer.toml scaffold to support tikv store 2021-08-26 16:26:27 +08:00
yulai.li 546efeba8f Fix build bug 2021-08-26 16:20:35 +08:00
yulai.li 2088f28424 init post 2021-08-26 15:20:18 +08:00
Chris Lu c08ac536ed cloud drive: add support for Wasabi 
* disable md5, sha256 checking to avoid reading one chunk twice
* single threaded upload to avoid chunk swapping (to be enhanced later)
 2021-08-25 17:34:29 -07:00
Chris Lu 9bcf94b2b1 ensure multi-threaded correctness 2021-08-25 17:28:50 -07:00
Chris Lu a7a914f120 cloud drive: add support for Baidu BOS 2021-08-24 23:46:33 -07:00
Chris Lu 2d805beed4 add help mesages 2021-08-24 23:28:35 -07:00
Chris Lu c6c97bd83d add default env variables 2021-08-24 23:25:36 -07:00
Chris Lu a19c728034 cloud drive: add support for Tencent COS 2021-08-24 23:19:45 -07:00
Chris Lu 19a81d25af cloud drive: add support for Aliyun OSS 2021-08-24 23:14:24 -07:00
Chris Lu 47d775cf68 cloud drive: add support for BackBlaze 2021-08-24 22:30:06 -07:00
Chris Lu 6bab20d862 cat entry content 2021-08-24 02:38:32 -07:00
Chris Lu e9ebe24f2e cloud drive: add support for Azure 2021-08-24 01:18:30 -07:00
Chris Lu 7c39a18ba5 update azure library 2021-08-24 00:32:35 -07:00
Chris Lu a78d0227cd adjust package name 2021-08-23 23:19:31 -07:00
Chris Lu 12631a3f5b cloud drive: gcs simplify a little bit 2021-08-23 14:43:01 -07:00
Chris Lu 00c4e06caa cloud drive: s3 configurable force path style 2021-08-23 03:30:41 -07:00
Chris Lu f0cc130849 do not force path style for better compatibility 2021-08-23 03:09:41 -07:00
Chris Lu 2836a58d87 cloud drive: S3 supports storage class 2021-08-23 02:18:59 -07:00
Chris Lu 95e2b83ca5 fix format 2021-08-23 00:49:59 -07:00
Chris Lu ff7dc3b44c 2.64 2021-08-23 00:39:15 -07:00
Chris Lu ca35a77e85 adjust help message 2021-08-23 00:37:55 -07:00
Chris Lu 258063de26 cloud drive: add google cloud storage 2021-08-23 00:29:27 -07:00
Chris Lu 3b2c39f1be mount: set name when mount path equals mount folder 
fix https://github.com/chrislusf/seaweedfs/issues/2275#issuecomment-903255876
 2021-08-22 04:59:09 -07:00
Chris Lu 2158d4fe4d adjust help message 2021-08-21 02:17:10 -07:00
Chris Lu 1b1ab331f6
Merge pull request #2274 from qieqieplus/gzip-pool 2021-08-20 04:22:41 -07:00
qieqieplus 7720533f84 reduce gzip allocation 2021-08-20 18:38:18 +08:00
Chris Lu 05fc7db755 filer.copy: large files pack file ids into chunks 2021-08-18 23:46:54 -07:00
mauroparente c63713774b
Fix image resize 
https://github.com/chrislusf/seaweedfs/issues/2262
 2021-08-18 23:15:35 +02:00
Chris Lu 570d2eccec skip body if not allowed by http status 2021-08-18 00:56:35 -07:00
Chris Lu ccb3df41f8 elide secret key 2021-08-17 11:27:08 -07:00
Konstantin Lebedev 865b06b7fe The IEEE assembler optimizations has been submitted and will be part of the Go 1.6 standard library. 
https://github.com/klauspost/crc32
 2021-08-17 13:06:48 +05:00
Chris Lu e2aa3cf63b fix go test 2021-08-15 23:20:46 -07:00
Chris Lu 1f35d32be0 2.63 2021-08-15 23:14:59 -07:00
Chris Lu 40dc283b2d fix locating data chunks 2021-08-15 23:07:58 -07:00
Chris Lu 72eb6d5b9d ensure no writes to remote storage if content is not changed 2021-08-15 20:23:41 -07:00
Chris Lu 5a7c40510f format output 2021-08-15 20:07:13 -07:00
Chris Lu bb94930196 add some delays if error 2021-08-15 20:06:47 -07:00
Chris Lu 5d5a21ba2d adjust log format 2021-08-15 19:46:45 -07:00
Chris Lu c45c12aedc metadata updates 2021-08-15 19:27:30 -07:00
Chris Lu 507411bdf0 fix https://github.com/chrislusf/seaweedfs/issues/2260 
fix https://github.com/chrislusf/seaweedfs/issues/2260
 2021-08-15 19:16:00 -07:00
Chris Lu c3ffd457ef fix compilation error 2021-08-15 12:40:22 -07:00
Chris Lu 49b5e47bd1 retry forever with filer.remote.sync, and some refactoring 2021-08-15 12:38:26 -07:00
Chris Lu fda2fc47b1 add RetryForever 2021-08-15 12:37:35 -07:00
Chris Lu a539d64896 refactor 2021-08-15 12:09:54 -07:00
Chris Lu 8f7d2d317f readerAt need to use the right offset 
fix https://github.com/chrislusf/seaweedfs/issues/2259
 2021-08-15 11:55:58 -07:00
Chris Lu 0b2a92d371 skip if http.StatusNoContent 2021-08-15 11:27:49 -07:00
Chris Lu ec989b0377 remove println 2021-08-15 02:10:27 -07:00
Chris Lu 9462f5129a shell: add "remote.meta.sync" 2021-08-15 01:53:46 -07:00
Chris Lu 3ada61c875 rename 2021-08-14 21:50:35 -07:00
Chris Lu c34747c79d rename, fix wrong logic. 2021-08-14 21:46:34 -07:00
Chris Lu cb53802752 adjust help message 2021-08-14 15:55:53 -07:00
Chris Lu 889b143fa7 adjust modification detection logic 2021-08-14 15:44:47 -07:00
Chris Lu f365af81c2 parallelize remote content fetching 2021-08-14 15:41:37 -07:00
Chris Lu 53e66980b2 add comments 2021-08-14 15:16:10 -07:00
Chris Lu 9921801e0c Revert "use default or path-specific setting for cache replication level" 
This reverts commit ba6923b223.
 2021-08-14 15:14:26 -07:00
Chris Lu ba6923b223 use default or path-specific setting for cache replication level 2021-08-14 15:14:01 -07:00
Chris Lu 708debca14 remote.cache and uncache: more flexible options to select files to cache or uncache 2021-08-14 15:11:55 -07:00
Chris Lu 08258a819d fix mistake 2021-08-14 05:10:30 -07:00
Chris Lu 4909bd9684 gRpc connection error on filer when no volume left #2243 
fix https://github.com/chrislusf/seaweedfs/issues/2243

grpc do not cache connections only when connection problem happens.
Normal error results should not close the shared grpc connection.
 2021-08-14 05:06:44 -07:00
Chris Lu 8126ab4b5d rename 2021-08-14 05:03:45 -07:00
Chris Lu 0db2517994 go fmt 2021-08-14 02:55:44 -07:00
Chris Lu 7937db52e1 Filer locationPrefix configure does not exec replication #2257 
fix https://github.com/chrislusf/seaweedfs/issues/2257
 2021-08-14 02:54:13 -07:00
Chris Lu 0c66b173a4 fix 2021-08-13 11:31:43 -07:00
Chris Lu e02a317d3d adjust retry logic in case some data is partially written 2021-08-13 11:30:38 -07:00
Chris Lu 2d519c6cb6 adjust the retry logic 2021-08-13 11:13:30 -07:00
Chris Lu b961fcd338 filer: stream read from volume server, reduce memory usage 2021-08-13 11:00:11 -07:00
Chris Lu f4decf02df volume copying: clean up stale volume data files 
fix https://github.com/chrislusf/seaweedfs/issues/2250
 2021-08-13 03:24:21 -07:00
Chris Lu 0f7d4556d8 shell: volume.tier.move makes up changes if volume move failed 2021-08-13 03:09:28 -07:00
Chris Lu 333cdce485 add verbose message 2021-08-13 02:57:14 -07:00
Chris Lu 78e8ddf910 Only when tailing volume, the zero-ed cookie should skip checking. 
This only happens when checkCookie == false and fsync == false.
 2021-08-13 02:09:35 -07:00
Chris Lu a8617c1a39 tail volume: fix zero cookie problem from batch deletion 2021-08-13 01:54:35 -07:00
Chris Lu 5a0f92423e use grpc and jwt 2021-08-12 21:40:33 -07:00
Chris Lu 6238644c35 remove gateway command 2021-08-12 20:52:04 -07:00
Chris Lu d1d1fc772c move some volume lookup operations to grpc 
jwt related lookup will come in next commit
 2021-08-12 20:33:00 -07:00
Chris Lu 5571f4f70a master: add master.follower to handle read file id lookup requests 2021-08-12 18:10:59 -07:00
Chris Lu 5469019852 adjust data type 2021-08-12 17:54:34 -07:00
Chris Lu bfac55e6c0 avoid integer overflow 
fix https://github.com/chrislusf/seaweedfs/issues/2254
 2021-08-11 22:22:49 -07:00