From 74e3e68a4dd39525d1b38ab4053a8808545985e9 Mon Sep 17 00:00:00 2001
From: Chris Lu <chrislusf@users.noreply.github.com>
Date: Thu, 10 Sep 2020 00:52:51 -0700
Subject: [PATCH] Updated Filer Store Replication (markdown)

---
 Filer-Store-Replication.md | 54 ++++++++++++++++++++++++++++++++++++++
 1 file changed, 54 insertions(+)

diff --git a/Filer-Store-Replication.md b/Filer-Store-Replication.md
index 729b57f..d235f3d 100644
--- a/Filer-Store-Replication.md
+++ b/Filer-Store-Replication.md
@@ -27,3 +27,57 @@ weed filer -port=8889 -peers=localhost:8888
 weed filer -port=8888 -peers=localhost:8888,localhost:8889
 weed filer -port=8889 -peers=localhost:8888,localhost:8889
 ```
+
+
+# Example Topologies
+
+* Multiple filers with leveldb stores
+
+```
+filer1(leveldb) <-> filer2(leveldb) <-> filer3(leveldb) 
+
+weed filer -peers=<filer1:port1>,<filer2:port2>,<filer3:port3>
+
+```
+
+* Two filers are fine. There is no requirements for number of filers.
+
+```
+filer1(leveldb) <-> filer2(leveldb)
+
+weed filer -peers=<filer1:port1>,<filer2:port2>
+
+```
+
+* Two filers with different stores are also fine. Of course, you will need a different `filer.toml`.
+
+```
+filer1(leveldb) <-> filer2(elastic search)
+
+weed filer -peers=<filer1:port1>,<filer2:port2>
+
+```
+
+* Master-Slave mode for filers with different stores.
+
+```
+filer1(leveldb) --> filer2(elastic search)
+
+# start filer2 as this. 
+weed filer -peers=<filer1:port1>
+
+```
+
+# How is it implemented?
+
+Each filer has a local meta data change log. When starting with `-peers` setting, each filer will subscribe to meta data changes from its peers and apply to local filer store.
+
+Each filer store will auto generate a unique `filer.store.id`. So for shared filer stores, such as mysql/postgres/redis, there is no need to setup peers because the `filer.store.id` will be the same.
+
+The subscription will also periodically checkpoint the subscription progress, so the subscription can resume if either filer is restarted.
+
+It is actually OK if you need to change filer IP or port. The replication can still resume as long as the filer store has the same content.
+
+# Limitations
+
+Multiple filers with local leveldb filer stores can work well with the `-peers` configured. However, this layout does not work well with `weed filer.sync` cross data center replication as of now. This is because currently `weed filer.sync` use `filer.store.id` to identify data that needs to be replicated. Having multiple `filer.store.id` will confuse the `weed filer.sync`.