From 4ddd71042a445e64340fe120e5ccf7aa6d13b509 Mon Sep 17 00:00:00 2001 From: Chris Lu Date: Sat, 25 Jul 2020 16:49:52 -0700 Subject: [PATCH] Updated Hadoop Benchmark (markdown) --- Hadoop-Benchmark.md | 22 +++++++++++----------- 1 file changed, 11 insertions(+), 11 deletions(-) diff --git a/Hadoop-Benchmark.md b/Hadoop-Benchmark.md index 7a26009..08cc6d9 100644 --- a/Hadoop-Benchmark.md +++ b/Hadoop-Benchmark.md @@ -39,18 +39,18 @@ The TestDFSIO benchmark is used for measuring I/O (read/write) performance. Start the TestDFSIO write tests: ``` -bin/hadoop jar ./share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-2.10.0-tests.jar TestDFSIO -write -nrFiles 8 -fileSize 1GB -resFile /tmp/TestDFSIOwrite.txt +bin/hadoop jar ./share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-2.10.0-tests.jar TestDFSIO -write -nrFiles 8 -size 32GB -bufferSize 8388608 -resFile /tmp/TestDFSIOwrite.txt ... -20/07/17 15:57:08 INFO fs.TestDFSIO: ----- TestDFSIO ----- : write -20/07/17 15:57:08 INFO fs.TestDFSIO: Date & time: Fri Jul 17 15:57:08 PDT 2020 -20/07/17 15:57:08 INFO fs.TestDFSIO: Number of files: 8 -20/07/17 15:57:08 INFO fs.TestDFSIO: Total MBytes processed: 8192 -20/07/17 15:57:08 INFO fs.TestDFSIO: Throughput mb/sec: 379.29 -20/07/17 15:57:08 INFO fs.TestDFSIO: Average IO rate mb/sec: 389.95 -20/07/17 15:57:08 INFO fs.TestDFSIO: IO rate std deviation: 74.91 -20/07/17 15:57:08 INFO fs.TestDFSIO: Test exec time sec: 24.05 -20/07/17 15:57:08 INFO fs.TestDFSIO: +20/07/25 16:48:21 INFO fs.TestDFSIO: ----- TestDFSIO ----- : read +20/07/25 16:48:21 INFO fs.TestDFSIO: Date & time: Sat Jul 25 16:48:21 PDT 2020 +20/07/25 16:48:21 INFO fs.TestDFSIO: Number of files: 8 +20/07/25 16:48:21 INFO fs.TestDFSIO: Total MBytes processed: 262144 +20/07/25 16:48:21 INFO fs.TestDFSIO: Throughput mb/sec: 399.16 +20/07/25 16:48:21 INFO fs.TestDFSIO: Average IO rate mb/sec: 399.34 +20/07/25 16:48:21 INFO fs.TestDFSIO: IO rate std deviation: 8.56 +20/07/25 16:48:21 INFO fs.TestDFSIO: Test exec time sec: 659.45 +20/07/25 16:48:21 INFO fs.TestDFSIO: ``` ## TestDFSIO read tests @@ -58,7 +58,7 @@ bin/hadoop jar ./share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-2.10.0 Start the TestDFSIO read tests: ``` -bin/hadoop jar ./share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-2.10.0-tests.jar TestDFSIO -read -nrFiles 8 -fileSize 1GB -resFile /tmp/TestDFSIOwrite.txt +bin/hadoop jar ./share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-2.10.0-tests.jar TestDFSIO -read -nrFiles 8 -size 32GB -bufferSize 8388608 -resFile /tmp/TestDFSIOwrite.txt ...