From 172d34a486962092d7551a07950f28c180bfe119 Mon Sep 17 00:00:00 2001 From: Chris Lu Date: Wed, 5 Jun 2019 13:39:37 -0700 Subject: [PATCH] Updated Erasure coding for warm storage (markdown) --- Erasure-coding-for-warm-storage.md | 109 +++++++++++++++++++++++++++++ 1 file changed, 109 insertions(+) diff --git a/Erasure-coding-for-warm-storage.md b/Erasure-coding-for-warm-storage.md index 4704e14..64aa2d6 100644 --- a/Erasure-coding-for-warm-storage.md +++ b/Erasure-coding-for-warm-storage.md @@ -34,3 +34,112 @@ When all data shards are online, the read are randomly assigned to one volume se In case of missing data shards or read failures from server B, server A will try to collect as many pieces of data as possible from the remaining servers, and compute the required data. ## Read Performance + +For this test, I started 4 volume servers. With each volume divided into 14 shards, + +Normal read performance by `weed benchmark -master localhost:9334 -n 102040 -collection=t -write=true` +``` +------------ Randomly Reading Benchmark ---------- +Completed 30822 of 102040 requests, 30.2% 30815.7/s 31.0MB/s +Completed 62311 of 102040 requests, 61.1% 31478.1/s 31.7MB/s +Completed 94190 of 102040 requests, 92.3% 31894.3/s 32.1MB/s + +Concurrency Level: 16 +Time taken for tests: 3.246 seconds +Complete requests: 102040 +Failed requests: 0 +Total transferred: 107698394 bytes +Requests per second: 31435.64 [#/sec] +Transfer rate: 32401.20 [Kbytes/sec] + +Connection Times (ms) + min avg max std +Total: 0.1 0.4 40.5 0.5 + +Percentage of the requests served within a certain time (ms) + 50% 0.3 ms + 66% 0.4 ms + 75% 0.5 ms + 90% 0.6 ms + 95% 0.9 ms + 98% 1.3 ms + 99% 1.8 ms + 100% 40.5 ms +``` + +After I force to erasure encode the volumes by `ec.encode -collection t -quietFor 1s -fullPercent=0.001` +Normal EC read performance by `weed benchmark -master localhost:9334 -n 102040 -collection=t -write=false`. You may need to run it twice because of some one-time read for the volume version. The EC read performance is about half of the normal volume read, because of the extra network hop. + +``` +------------ Randomly Reading Benchmark ---------- +Completed 14077 of 102040 requests, 13.8% 14046.8/s 14.1MB/s +Completed 28261 of 102040 requests, 27.7% 14184.1/s 14.3MB/s +Completed 42580 of 102040 requests, 41.7% 14348.0/s 14.4MB/s +Completed 56617 of 102040 requests, 55.5% 14008.5/s 14.1MB/s +Completed 70513 of 102040 requests, 69.1% 13896.0/s 14.0MB/s +Completed 84264 of 102040 requests, 82.6% 13751.1/s 13.8MB/s +Completed 97858 of 102040 requests, 95.9% 13623.1/s 13.7MB/s + +Concurrency Level: 16 +Time taken for tests: 7.306 seconds +Complete requests: 102040 +Failed requests: 0 +Total transferred: 107699432 bytes +Requests per second: 13966.69 [#/sec] +Transfer rate: 14395.82 [Kbytes/sec] + +Connection Times (ms) + min avg max std +Total: 0.1 1.1 24.9 0.7 + +Percentage of the requests served within a certain time (ms) + 50% 1.0 ms + 66% 1.2 ms + 75% 1.3 ms + 80% 1.4 ms + 90% 1.7 ms + 95% 2.1 ms + 98% 2.8 ms + 99% 3.4 ms + 100% 24.9 ms +``` + +Now let's stop one server, which has 3 or 4 data shards on it. There is a costly read repair required for about 28.6% of the data. + +``` +------------ Randomly Reading Benchmark ---------- +Completed 9458 of 102040 requests, 9.3% 9456.8/s 9.5MB/s +Completed 19216 of 102040 requests, 18.8% 9758.1/s 9.8MB/s +Completed 28837 of 102040 requests, 28.3% 9620.1/s 9.7MB/s +Completed 38221 of 102040 requests, 37.5% 9385.6/s 9.4MB/s +Completed 47410 of 102040 requests, 46.5% 9177.2/s 9.2MB/s +Completed 56586 of 102040 requests, 55.5% 9186.0/s 9.2MB/s +Completed 66274 of 102040 requests, 64.9% 9679.4/s 9.7MB/s +Completed 75385 of 102040 requests, 73.9% 9120.8/s 9.2MB/s +Completed 84028 of 102040 requests, 82.3% 8643.3/s 8.7MB/s +Completed 92447 of 102040 requests, 90.6% 8416.7/s 8.5MB/s +Completed 100831 of 102040 requests, 98.8% 8386.2/s 8.4MB/s + +Concurrency Level: 16 +Time taken for tests: 11.149 seconds +Complete requests: 102040 +Failed requests: 0 +Total transferred: 107702392 bytes +Requests per second: 9152.52 [#/sec] +Transfer rate: 9433.99 [Kbytes/sec] + +Connection Times (ms) + min avg max std +Total: 0.1 1.7 40.1 1.6 + +Percentage of the requests served within a certain time (ms) + 50% 1.2 ms + 66% 1.7 ms + 75% 2.1 ms + 80% 2.4 ms + 90% 3.5 ms + 95% 4.6 ms + 98% 6.4 ms + 99% 7.7 ms + 100% 40.1 ms +``` \ No newline at end of file