Commit graph

387 commits

Author SHA1 Message Date
Chris Lu dc481c081c remove unused function 2021-08-27 22:18:54 -07:00
Chris Lu 9242f3aaeb fix build 2021-08-26 17:33:57 -07:00
Chris Lu 05a648bb96 refactor: separating out remote.proto 2021-08-26 15:18:34 -07:00
Chris Lu 9bcf94b2b1 ensure multi-threaded correctness 2021-08-25 17:28:50 -07:00
Chris Lu 40dc283b2d fix locating data chunks 2021-08-15 23:07:58 -07:00
Chris Lu 72eb6d5b9d ensure no writes to remote storage if content is not changed 2021-08-15 20:23:41 -07:00
Chris Lu 5d5a21ba2d adjust log format 2021-08-15 19:46:45 -07:00
Chris Lu 49b5e47bd1 retry forever with filer.remote.sync, and some refactoring 2021-08-15 12:38:26 -07:00
Chris Lu 8f7d2d317f readerAt need to use the right offset
fix https://github.com/chrislusf/seaweedfs/issues/2259
2021-08-15 11:55:58 -07:00
Chris Lu 9462f5129a shell: add "remote.meta.sync" 2021-08-15 01:53:46 -07:00
Chris Lu 0c66b173a4 fix 2021-08-13 11:31:43 -07:00
Chris Lu e02a317d3d adjust retry logic in case some data is partially written 2021-08-13 11:30:38 -07:00
Chris Lu 2d519c6cb6 adjust the retry logic 2021-08-13 11:13:30 -07:00
Chris Lu b961fcd338 filer: stream read from volume server, reduce memory usage 2021-08-13 11:00:11 -07:00
Chris Lu 5a0f92423e use grpc and jwt 2021-08-12 21:40:33 -07:00
Chris Lu bfac55e6c0 avoid integer overflow
fix https://github.com/chrislusf/seaweedfs/issues/2254
2021-08-11 22:22:49 -07:00
Chris Lu 9d85569c55 ensure using local quorum consistency 2021-08-10 05:10:57 -07:00
Chris Lu 713c035a6e shell: remote.cache remote.uncache 2021-08-09 14:35:18 -07:00
Chris Lu c5f38c365d go fmt 2021-08-08 22:30:36 -07:00
Chris Lu c0b12da4ef shell: add filer.remote.unmount 2021-08-08 22:26:37 -07:00
Chris Lu 13e45e1605 filer.remote.sync can work now 2021-08-08 01:21:42 -07:00
Chris Lu 46b0cb8c86 Merge branch 'master' into add_remote_storage 2021-08-07 15:41:27 -07:00
Chris Lu de730b079d ChunkStreamReader implenents io.ReaderAt 2021-08-07 15:41:07 -07:00
Chris Lu 59732a0529 refactoring 2021-08-07 15:35:27 -07:00
Chris Lu ecb234f75a refactor 2021-08-07 14:46:23 -07:00
Chris Lu 679f800caa Merge branch 'master' into add_remote_storage 2021-08-06 20:27:12 -07:00
byunghwa.yun f3dc909b21 Change default permissions 2021-08-06 12:24:35 +09:00
Chris Lu 6b743dbbf9 refactor client subscribe metadata 2021-08-04 16:25:46 -07:00
Chris Lu f6a9ad8001 fix tests 2021-08-04 00:31:06 -07:00
Chris Lu 81b255df8b Merge branch 'master' into add_remote_storage 2021-08-01 15:44:21 -07:00
Chris Lu 89933c46d2 s3: skip hidden directories in /buckets folder 2021-08-01 12:28:08 -07:00
Chris Lu 767edd3c08 rename 2021-07-31 23:52:09 -07:00
Chris Lu 9df7d16791 read <- remote_storage 2021-07-31 22:39:38 -07:00
Chris Lu 899963ac20 remote storage location changed to struct 2021-07-29 02:08:55 -07:00
Chris Lu c090d6bb25 add ReadRemote(), add read remote setup when filer starts 2021-07-28 22:43:12 -07:00
Chris Lu 1752eeb538 remote.mount saves the mapping 2021-07-27 03:26:35 -07:00
Chris Lu 4b94b03d90 directory to remote storage mapping 2021-07-27 01:16:28 -07:00
Chris Lu 99b599aa8a remote.mount 2021-07-26 22:53:44 -07:00
byunghwa.yun 2b28a818f1 Fix mysql sql for batch delete 2021-07-25 11:06:14 +09:00
Chris Lu bdb632fa62
Merge pull request #2211 from qieqieplus/filer-bloom-filter
add bloom filter for filer leveldb/rocksdb
2021-07-22 23:35:54 -07:00
qieqieplus 2b46df13f8 add bloom filter for filer leveldb/rocksdb 2021-07-23 14:05:59 -06:00
Chris Lu 182288f860 filer: fix mysql, postgres batch delete error 2021-07-22 08:23:20 -07:00
Chris Lu 7359193e97 go fmt 2021-07-21 14:38:12 -07:00
Chris Lu 70effac0d3 configure and store remote configurations 2021-07-21 02:24:34 -07:00
Chris Lu a125c8fbe2
Merge pull request #2204 from nivekuil/tokenpolicy
cassandra: use LocalQuorum for all queries
2021-07-20 15:37:31 -07:00
nivekuil b9a67d46c5 cassandra: use LocalQuorum for all queries
This changes this filer store from eventual to strong consistency at the cost
of read performance.
2021-07-20 14:47:39 -07:00
Chris Lu 54c8bc8673 Merge branch 'remote_overlay' 2021-07-19 23:18:35 -07:00
Chris Lu 7ab389e7ec optimization: improve random range query for large files 2021-07-19 23:07:22 -07:00
Chris Lu 93c37cfded
Update filechunk_manifest.go 2021-07-19 18:41:41 -07:00
Chris Lu 450222dd64 add remote to filer.Entry and filer_pb entry, add RemoteConf 2021-07-19 02:47:27 -07:00