gitlab-com-seaweedfs/seaweedfs
1
0
Fork 0
mirror of https://github.com/seaweedfs/seaweedfs.git synced 2024-01-19 02:48:24 +00:00
Code Issues Projects Releases Wiki Activity

Created Async Filer Metadata Backup (markdown)

Chris Lu 2021-03-03 22:07:25 -08:00
parent 74754ec331
commit 637a22c409
1 changed files with 25 additions and 0 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

25
Async-Filer-Metadata-Backup.md Normal file

@ -0,0 +1,25 @@
It is desirable to have a copy of the filer metadata. However, it is not always easy to setup replications for filer metadata store:
* The store does support stream replication.
* The store is embedded leveldb, RocksDB, etc.
* It is just complicated.
There are two ways to ensure a copy of the meta data.
# Separate Filer instances connected with "-peers" option
With `-peers` option, if the filers are not sharing the same filer metadata store, the metadata changes are asynchronously propagated to all peers.
See https://github.com/chrislusf/seaweedfs/wiki/Filer-Store-Replication#file-store-replication
# Async filer metadata backup to a remote store
We can also continuously backup filer metadata to a remote store, without running a filer instance.
```
Metadata Changes
Filer --------------------> `weed filer.meta.backup` ----> Remote Store(LevelDB/RocksDB/
Mysql/Postgres/
Redis/Cassanra/...)
```
Just need to configure the remote store the same way as `filer.toml`.
`weed filer.meta.backup` can be stopped and resumed. The metadata backup progress is tracked in the remote store itself. So you can pause/resume any time, and even resume from a separate machine.