Commit graph

56 commits

Author SHA1 Message Date
Konstantin Lebedev 1169f94310
Fix filer sync set offset (#5197)
* fix: compose 2mount with sync

* fix: DATA RACE
https://github.com/seaweedfs/seaweedfs/issues/5194
https://github.com/seaweedfs/seaweedfs/issues/5195
2024-01-12 10:57:18 -08:00
Konstantin Lebedev b9d32d32e1
chore: filer sync add doDeleteFiles option for create only mode (#5166) 2024-01-06 10:02:16 -08:00
Konstantin Lebedev 2b229e98ce fix: add doDeleteFile option for filer backup 2023-11-17 07:37:28 -08:00
Konstantin Lebedev 3c5295a1a6 filer backup add option for exclude file names that match the regexp to sync on filer 2023-11-13 06:23:46 -08:00
chrislu 5db9fcccd4 refactoring 2023-03-21 23:01:49 -07:00
chrislu 81fdf3651b grpc connection to filer add sw-client-id header 2023-01-20 01:48:12 -08:00
chrislu 8a40fa8993 more detailed logs 2022-12-17 13:18:35 -08:00
Jiffs Maverick 4b0430e71d
[metrics] Add the ability to control bind ip (#4012) 2022-11-24 10:22:59 -08:00
chrislu 21c0587900 go fmt 2022-09-14 23:06:44 -07:00
Ryan Russell 2c92a9ff74
refactor: DefaultConcurrencyLimit var rename (#3658) 2022-09-14 06:30:32 -07:00
bernardx 228b133afa
new 'concurrency' parameter for filer.sync (#3579)
Co-authored-by: XIAOYQ <xiaoyq@eudic.net>
2022-09-02 23:03:23 -07:00
qzh 7fcfaf7bc9
fix(filer.sync): offset may be set to 0 (#3451)
* fix(filer.sync): initializing the offset is related to the path

* fix(filer.sync): the offset maybe to be set to 0.

Co-authored-by: zhihao.qu <zhihao.qu@ly.com>
2022-08-15 23:43:52 -07:00
qzh 400f0c3e5d
fix(filer.sync): initializing the offset is related to the path (#3450)
Co-authored-by: zhihao.qu <zhihao.qu@ly.com>
2022-08-15 21:56:47 -07:00
chrislu 67814a5c79 refactor and fix strings.Split 2022-08-07 01:34:32 -07:00
chrislu 1a4bf0dcb5 filer.sync: parallelize the filer.sync 2022-08-07 00:56:15 -07:00
chrislu 0e9478488d filer.sync: fix when excluded paths is empty 2022-08-07 00:55:34 -07:00
chrislu 26dbc6c905 move to https://github.com/seaweedfs/seaweedfs 2022-07-29 00:17:28 -07:00
Konstantin Lebedev 7e09a548a6 exclude directories to sync on filer 2022-07-27 19:22:57 +05:00
chrislu 64f3d6fb6e metadata subscription uses client epoch 2022-07-23 10:50:28 -07:00
chrislu 4a65159250 fix reading time 2022-06-27 12:40:47 -07:00
zhihao.qu 4d0d1848c6 fix(filer.sync): modify clientName format : from -> to 2022-06-15 13:33:20 +08:00
zhihao.qu 42d04c581b feat(filer.sync): add metricsServer in filer.sync.
Metrics include:
(1) the offset of the filer.sync
(2) the last send timestamp of the filer subscription
2022-06-15 11:33:18 +08:00
zhihao.qu 14d82c3dea feat(filer.sync): add offset to path. 2022-06-14 19:46:02 +08:00
zhihao.qu cd5cca36a4 feat(filer.sync): add fromTsMs. Extract signature from doSubscribeFilerMetaChanges 2022-06-09 10:53:19 +08:00
creeew 02ae102731 fix filer.sync missing source srv uploaded files to target when target down 2022-06-02 01:28:47 +08:00
chrislu a2b101a737 subscribe metadata between a range 2022-05-30 15:04:19 -07:00
chrislu 1384529eb7 Fix filer.backup deletes files in backup folder in incremental mode
fix https://github.com/chrislusf/seaweedfs/issues/2919
2022-04-14 13:35:01 -07:00
chrislu 202a29d014 refactoring 2022-02-25 01:17:26 -08:00
chrislu 6a12520a96 fix logging 2022-01-10 01:00:11 -08:00
chrislu 5c87fcc6d2 add client id for all metadata listening clients 2021-12-30 00:23:57 -08:00
chrislu 9f9ef1340c use streaming mode for long poll grpc calls
streaming mode would create separate grpc connections for each call.
this is to ensure the long poll connections are properly closed.
2021-12-26 00:15:03 -08:00
Chris Lu e5fc35ed0c change server address from string to a type 2021-09-12 22:47:52 -07:00
Chris Lu 8e125339d5 line wrap 2021-08-31 23:30:28 -07:00
Chris Lu 68ea99190f add additionalPathPrefixes 2021-08-31 23:26:26 -07:00
Chris Lu c5f38c365d go fmt 2021-08-08 22:30:36 -07:00
Chris Lu df85f7a1eb adjust help message 2021-08-08 22:30:12 -07:00
Chris Lu 6b743dbbf9 refactor client subscribe metadata 2021-08-04 16:25:46 -07:00
Chris Lu 3afbf04007 add TLS grpc support for filer meta clients 2021-08-03 01:30:35 -07:00
Chris Lu c2e0a75c1f adjust logs 2021-06-24 12:46:00 -07:00
Chris Lu 2d7b4e5bb6 filer.backup: escape colon from path on windows
fix https://github.com/chrislusf/seaweedfs/issues/2084
2021-05-29 06:45:27 -07:00
Chris Lu e52c94640e filer.backup: added to replace filer.replicate 2021-02-28 16:22:27 -08:00
Chris Lu 62191b08ea disk type support custom tags 2021-02-22 02:03:12 -08:00
Chris Lu 0bc3a1f9e8 disk type only supports hdd and ssd, not ready for random tags yet 2021-02-14 11:38:43 -08:00
Chris Lu ef76365ec2 adjust help message 2021-02-13 15:47:08 -08:00
Chris Lu 821c46edf1 Merge branch 'master' into support_ssd_volume 2021-02-09 11:37:07 -08:00
Chris Lu d475c89fcc go fmt 2021-01-28 15:23:46 -08:00
Chris Lu cc1f3907ff filer.sync: default filerProxy to true 2021-01-25 10:14:42 -08:00
Chris Lu 80b8692688 filer.sync: replicate outside of either cluster, only need to see filers 2021-01-24 00:01:44 -08:00
Chris Lu 1bf22c0b5b go fmt 2020-12-16 09:14:05 -08:00
Chris Lu 8baba93fce rename parameter name to "disk" 2020-12-13 12:06:21 -08:00