From 3a427c8a8e0d36a42003555d6ce1f9b77d48067b Mon Sep 17 00:00:00 2001 From: Chris Lu Date: Thu, 6 Jun 2019 00:59:13 -0700 Subject: [PATCH] Updated Erasure coding for warm storage (markdown) --- Erasure-coding-for-warm-storage.md | 8 +++++--- 1 file changed, 5 insertions(+), 3 deletions(-) diff --git a/Erasure-coding-for-warm-storage.md b/Erasure-coding-for-warm-storage.md index 01cdca5..4482ee8 100644 --- a/Erasure-coding-for-warm-storage.md +++ b/Erasure-coding-for-warm-storage.md @@ -19,14 +19,16 @@ The scripts has 3 steps. The default command is `ec.encode -fullPercent=95 -quietFor=1h`. It will find volumes at least 95% of the maximum volume size, which is usually 30GB, and has no updates for 1 hour. ### Data Repair -The default command is `ec.rebuild -force`. If disk fails or server fails, some data shards are lost. With erasure coding, we can recover the lost data shards from remaining data shards. +If disk fails or server fails, some data shards are lost. With erasure coding, we can recover the lost data shards from remaining data shards. -The data repair happens for the whole volume, instead of one file at a time. It is much more efficient. +The default command is `ec.rebuild -force`. + +The data repair happens for the whole volume, instead of one small file at a time. It is much more efficient and fast to reconstruct the missing data shards. ### EC data balancing With servers added or removed, some data shards may not be laid out optimally. For example, one volume's 5 data shards could be on the same server. If the server goes down, the volume would be unrepairable or part of the data is lost permanently. -The default command is `ec.balance -force`. It will try to spread the data shards to minimize the data loss risk. +The default command is `ec.balance -force`. It will try to spread the data shards evenly to minimize the data shard loss risk. ## How the read works? When all data shards are online, the read are randomly assigned to one volume server (A) that has at least one data shard. Server A will read its copy of index file, and locate the volume server (B), and read from server B for the file.