Commit graph

103 commits

Author SHA1 Message Date
Chris Lu 9a06c35da4 replicate: incremental sink only contains new and updated files
address da08402ba2
2021-01-28 02:39:22 -08:00
Chris Lu da08402ba2 replicate: use creation time for local incremental file sink
related to https://github.com/chrislusf/seaweedfs/pull/1762
2021-01-28 02:17:41 -08:00
Konstantin Lebedev 02fdc0a333 rename backup to local_incremental and use mtime 2021-01-28 14:56:13 +05:00
Konstantin Lebedev 6b54ff9912 replication to create time date directory 2021-01-27 15:01:33 +05:00
Konstantin Lebedev 3634811408 replication to todays date directory 2021-01-27 12:45:58 +05:00
Chris Lu 5e07afb0f0
Merge pull request #1759 from kmlebedev/sink.local
replication to local disk storage
2021-01-26 22:31:54 -08:00
Chris Lu ad2a20c8a5 notification add ack and nack 2021-01-26 11:08:44 -08:00
Konstantin Lebedev 612b7975a1 replication to local disk storage 2021-01-26 22:50:25 +05:00
Chris Lu 00707ec00f mount: outsideContainerClusterMode proxy through filer
Running mount outside of the cluster would not need to expose all the volume servers to outside of the cluster. The chunk read and write will go through the filer.
2021-01-24 19:01:58 -08:00
Chris Lu 6ca10725b8 Revert "mount: when outside cluster network, use filer as proxy to access volume servers"
This reverts commit 096e088d7b.
2021-01-24 03:15:19 -08:00
Chris Lu 096e088d7b mount: when outside cluster network, use filer as proxy to access volume servers 2021-01-24 01:41:38 -08:00
Chris Lu 80b8692688 filer.sync: replicate outside of either cluster, only need to see filers 2021-01-24 00:01:44 -08:00
Chris Lu 2b76854641 add "weed filer.cat" to read files directly from volume servers 2021-01-06 04:22:00 -08:00
Chris Lu f4abd01adf filer: cache small file to filer store 2020-11-30 04:34:04 -08:00
Chris Lu e219c57849 passing full path when assign volume locations 2020-10-25 15:46:29 -07:00
Chris Lu f375b93aef renaming 2020-10-25 15:32:43 -07:00
Chris Lu d75bc62196 to read files that are quickly changing 2020-10-22 12:59:15 -07:00
Chris Lu 3f7d1d1bf1 Only wait on retryable requests 2020-10-13 00:29:46 -07:00
Chris Lu 723ae11db4 refactoring in order to adjust volume server url later 2020-10-11 20:15:10 -07:00
Chris Lu a8624c2e4f read from alternative replica
related to https://github.com/chrislusf/seaweedfs/issues/1512
2020-10-07 22:49:04 -07:00
Chris Lu 446e476a11 go fmt 2020-09-12 04:08:03 -07:00
Chris Lu 387ab6796f filer: cross cluster synchronization 2020-09-09 11:21:23 -07:00
Chris Lu 4fc0bd1a81 return http response directly 2020-09-09 03:53:09 -07:00
Chris Lu eb7929a971 rename filer2 to filer 2020-09-01 00:21:19 -07:00
Chris Lu ca658a97c5 add signatures to messages to avoid double processing 2020-08-28 23:48:48 -07:00
Chris Lu c27e18aa6a read possible old deleted chunks 2020-08-19 00:43:07 -07:00
Chris Lu c647deace1 file size support set file length
use Attr.FileSize and TotalChunkSize to determine file size
2020-08-15 09:32:47 -07:00
Chris Lu 97d97f3528 go code can read and write chunk manifest 2020-07-19 17:59:43 -07:00
Chris Lu 37d5b3ba12 replication: pass isFromOtherCluster also to EventNotification
EventNotification is consistent with message queue and metadata logs.
2020-07-01 08:06:20 -07:00
Chris Lu 31e23e9783 filer: support active<=>active filer replication 2020-06-30 22:53:57 -07:00
Chris Lu 97239ce6f1 rename filechunk is_gzipped to is_compressed 2020-06-20 08:15:49 -07:00
Chris Lu ed3cf811f5 refactoring 2020-04-29 13:26:02 -07:00
Chris Lu f282ed444b refactoring 2020-04-13 21:58:10 -07:00
Chris Lu 4b27afe176 fix sqs message reading 2020-04-10 15:27:47 -07:00
Chris Lu ec2eb8bc48 add If-None-Match and If-Modified-Since
fix https://github.com/chrislusf/seaweedfs/issues/1269
2020-04-08 08:12:00 -07:00
Chris Lu 4c498e73de filer replication: add s3 endpoint 2020-04-07 17:49:00 -07:00
Chris Lu 91da7057b1 refactoring 2020-04-05 13:11:43 -07:00
Chris Lu eedd33dda3 refactoring 2020-03-28 13:41:58 -07:00
Chris Lu b97768c51c refactoring 2020-03-23 01:30:22 -07:00
Chris Lu c0f0fdb3ba refactoring 2020-03-23 00:01:34 -07:00
Chris Lu ae2ee379c0 consistent 64bit size 2020-03-22 01:37:46 -07:00
Chris Lu bec6ec7db6 go fmt 2020-03-17 10:01:55 -07:00
HongyanShen 81610ed006 fix: #1226 2020-03-11 14:37:14 +08:00
Chris Lu 2e3f6ad3a9 filer: remember content is gzipped or not 2020-03-08 21:39:33 -07:00
Chris Lu 8645283a7b fuse mount: avoid lookup nil entry
fix https://github.com/chrislusf/seaweedfs/issues/1221
2020-03-07 16:51:46 -08:00
Chris Lu 13e215ee5c filer: option to encrypt data on volume server 2020-03-06 00:49:47 -08:00
Chris Lu f90c43635d refactoring 2020-03-04 00:39:47 -08:00
Chris Lu 97ab8a1976 remove ctx if possible 2020-02-25 22:23:59 -08:00
Chris Lu 892e726eb9 avoid reusing context object
fix https://github.com/chrislusf/seaweedfs/issues/1182
2020-02-25 21:50:12 -08:00
Chris Lu 0841bedb15 move filer assign volume grpc errror to response 2020-02-25 17:15:09 -08:00