From 17fcf828d35b09d4c139e36fd14956a19dfcbfc1 Mon Sep 17 00:00:00 2001 From: ernado Date: Wed, 23 Jul 2014 16:30:16 +0400 Subject: [PATCH] added replication docs --- docs/index.rst | 1 + docs/replication.rst | 98 ++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 99 insertions(+) create mode 100644 docs/replication.rst diff --git a/docs/index.rst b/docs/index.rst index 8dc87e8cb..93980ac45 100644 --- a/docs/index.rst +++ b/docs/index.rst @@ -26,6 +26,7 @@ Contents: gettingstarted clients api + replication Indices and tables diff --git a/docs/replication.rst b/docs/replication.rst new file mode 100644 index 000000000..136f41b97 --- /dev/null +++ b/docs/replication.rst @@ -0,0 +1,98 @@ +Replication +=================================== +Weed-FS can support replication. The replication is implemented not on file level, but on volume level. + +How to use +################################### +Basically, the way it works is: + +1. start weed master, and optionally specify the default replication type + +.. code-block:: bash + + ./weed master -defaultReplicationType=001 + +2. start volume servers as this: + +.. code-block:: bash + + ./weed volume -port=8081 -dir=/tmp/1 -max=100 + ./weed volume -port=8082 -dir=/tmp/2 -max=100 + ./weed volume -port=8083 -dir=/tmp/3 -max=100 + +Submitting, Reading, Deleting files has the same steps. + +The meaning of replication type +################################### +*Note: This subject to change.* + ++-----+---------------------------------------------------------------------------+ +|000 |no replication, just one copy | ++-----+---------------------------------------------------------------------------+ +|001 |replicate once on the same rack | ++-----+---------------------------------------------------------------------------+ +|010 |replicate once on a different rack in the same data center | ++-----+---------------------------------------------------------------------------+ +|100 |replicate once on a different data center | ++-----+---------------------------------------------------------------------------+ +|200 |replicate twice on two other different data center | ++-----+---------------------------------------------------------------------------+ +|110 |replicate once on a different rack, and once on a different data center | ++-----+---------------------------------------------------------------------------+ +|... |... | ++-----+---------------------------------------------------------------------------+ + +So if the replication type is xyz + ++-------+--------------------------------------------------------+ +|**x** |number of replica in other data centers | ++-------+--------------------------------------------------------+ +|**y** |number of replica in other racks in the same data center| ++-------+--------------------------------------------------------+ +|**z** |number of replica in other servers in the same rack | ++-------+--------------------------------------------------------+ + +x,y,z each can be 0, 1, or 2. So there are 9 possible replication types, and can be easily extended. +Each replication type will physically create x+y+z+1 copies of volume data files. + +Example topology configuration +################################### + +The WeedFS master server tries to read the default topology configuration file are read from /etc/weedfs/weedfs.conf, if it exists. The topology setting to configure data center and racks file format is as this. + +.. code-block:: xml + + + + + + 192.168.1.1 + + + + + 192.168.1.2 + + + 192.168.1.3 + 192.168.1.4 + + + + + +Allocate File Key on specific data center +Volume servers can start with a specific data center name. + +.. code-block:: bash + + weed volume -dir=/tmp/1 -port=8080 -dataCenter=dc1 + weed volume -dir=/tmp/2 -port=8081 -dataCenter=dc2 + +Or the master server can determine the data center via volume server's IP address and settings in weed.conf file. + +Now when requesting a file key, an optional "dataCenter" parameter can limit the assigned volume to the specific data center. For example, this specify + +.. code-block:: bash + + http://localhost:9333/dir/assign?dataCenter=dc1 \ No newline at end of file