gitlab-com-seaweedfs/seaweedfs
1
0
Fork 0
mirror of https://github.com/seaweedfs/seaweedfs.git synced 2024-01-19 02:48:24 +00:00
Code Issues Projects Releases Wiki Activity

Updated Erasure Coding for warm storage (markdown)

Chris Lu 2019-06-26 22:25:04 -07:00
parent b56d66c546
commit 10f200f625
1 changed files with 5 additions and 0 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

5
Erasure-Coding-for-warm-storage.md

@ -20,6 +20,11 @@ The downside:
* Only deletion is supported. Update is not supported.
* Compaction is WIP. This would require transmitting whole volume data.
Side Note:
* The 10+4 can be easily adjusted via `DataShardsCount` and `ParityShardsCount` in
https://github.com/chrislusf/seaweedfs/blob/master/weed/storage/erasure_coding/ec_encoder.go#L17
* If you are considering these enterprise-level customizations, please consider donate to SeaweedFS first.
## Architecture
SeaweedFS implemented 10.4 Reed-Soloman Erasure Coding (EC). The large volumes are split into chunks of 1GB, and every 10 data chunks are also encoded into 4 parity chunks. So a 30 GB data volume will be encoded into 14 EC shards, each shard is of size 3 GB and has 3 EC blocks.