gitlab-com-seaweedfs/seaweedfs
1
0
Fork 0
mirror of https://github.com/seaweedfs/seaweedfs.git synced 2024-01-19 02:48:24 +00:00
Code Issues Projects Releases Wiki Activity

Updated Hadoop Benchmark (markdown)

Chris Lu 2020-07-25 16:49:52 -07:00
parent f2241d6133
commit 4ddd71042a
1 changed files with 11 additions and 11 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

22
Hadoop-Benchmark.md

@ -39,18 +39,18 @@ The TestDFSIO benchmark is used for measuring I/O (read/write) performance.
Start the TestDFSIO write tests:
```
bin/hadoop jar ./share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-2.10.0-tests.jar TestDFSIO -write -nrFiles 8 -fileSize 1GB -resFile /tmp/TestDFSIOwrite.txt
bin/hadoop jar ./share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-2.10.0-tests.jar TestDFSIO -write -nrFiles 8 -size 32GB -bufferSize 8388608 -resFile /tmp/TestDFSIOwrite.txt
...
20/07/17 15:57:08 INFO fs.TestDFSIO: ----- TestDFSIO ----- : write
20/07/17 15:57:08 INFO fs.TestDFSIO: Date & time: Fri Jul 17 15:57:08 PDT 2020
20/07/17 15:57:08 INFO fs.TestDFSIO: Number of files: 8
20/07/17 15:57:08 INFO fs.TestDFSIO: Total MBytes processed: 8192
20/07/17 15:57:08 INFO fs.TestDFSIO: Throughput mb/sec: 379.29
20/07/17 15:57:08 INFO fs.TestDFSIO: Average IO rate mb/sec: 389.95
20/07/17 15:57:08 INFO fs.TestDFSIO: IO rate std deviation: 74.91
20/07/17 15:57:08 INFO fs.TestDFSIO: Test exec time sec: 24.05
20/07/17 15:57:08 INFO fs.TestDFSIO:
20/07/25 16:48:21 INFO fs.TestDFSIO: ----- TestDFSIO ----- : read
20/07/25 16:48:21 INFO fs.TestDFSIO: Date & time: Sat Jul 25 16:48:21 PDT 2020
20/07/25 16:48:21 INFO fs.TestDFSIO: Number of files: 8
20/07/25 16:48:21 INFO fs.TestDFSIO: Total MBytes processed: 262144
20/07/25 16:48:21 INFO fs.TestDFSIO: Throughput mb/sec: 399.16
20/07/25 16:48:21 INFO fs.TestDFSIO: Average IO rate mb/sec: 399.34
20/07/25 16:48:21 INFO fs.TestDFSIO: IO rate std deviation: 8.56
20/07/25 16:48:21 INFO fs.TestDFSIO: Test exec time sec: 659.45
20/07/25 16:48:21 INFO fs.TestDFSIO:
```
## TestDFSIO read tests
@ -58,7 +58,7 @@ bin/hadoop jar ./share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-2.10.0
Start the TestDFSIO read tests:
```
bin/hadoop jar ./share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-2.10.0-tests.jar TestDFSIO -read -nrFiles 8 -fileSize 1GB -resFile /tmp/TestDFSIOwrite.txt
bin/hadoop jar ./share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-2.10.0-tests.jar TestDFSIO -read -nrFiles 8 -size 32GB -bufferSize 8388608 -resFile /tmp/TestDFSIOwrite.txt
...