gitlab-com-seaweedfs/seaweedfs
1
0
Fork 0
mirror of https://github.com/seaweedfs/seaweedfs.git synced 2024-01-19 02:48:24 +00:00
Code Issues Projects Releases Wiki Activity

Updated FUSE Mount (markdown)

Chris Lu 2020-10-21 13:46:34 -07:00
parent e686e86102
commit 3b0fd3007c
1 changed files with 51 additions and 53 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

104
FUSE-Mount.md

@ -182,70 +182,68 @@ Threads fairness:
``` ```
Here is the result of using tool [fio](https://github.com/axboe/fio). Here is the result of using tool [fio](https://github.com/axboe/fio).
* The read performance is great because SeaweedFS FUSE asynchronously caches both metadata and file content. * The server and mount are restarted before each run.
* The write performance may seem low, because SeaweedFS FUSE chooses to always commit to the remote storage first, instead of writing to local cache and claim success too early. * Mount disabled the local cache: `weed mount -dir=xx -cacheCapacityMB=0`
``` ```
$ fio --name=sequential-read -directory=mm --rw=read --refill_buffers --bs=4M --size=4G ~$ # restart server and mount
sequential-read: (g=0): rw=read, bs=(R) 4096KiB-4096KiB, (W) 4096KiB-4096KiB, (T) 4096KiB-4096KiB, ioengine=psync, iodepth=1 ~$ fio --name=sequential-write -directory=/Users/chris/tmp/mm/buckets/large --rw=write --refill_buffers --bs=4M --size=1G
fio-3.19-68-gf09a77
Starting 1 process
sequential-read: Laying out IO file (1 file / 4096MiB)
Jobs: 1 (f=1): [R(1)][100.0%][r=965MiB/s][r=241 IOPS][eta 00m:00s]
sequential-read: (groupid=0, jobs=1): err= 0: pid=63589: Mon May 18 21:29:33 2020
read: IOPS=91, BW=367MiB/s (385MB/s)(4096MiB/11159msec)
clat (usec): min=456, max=338400, avg=10894.02, stdev=42980.01
lat (usec): min=457, max=338401, avg=10894.28, stdev=42979.99
clat percentiles (usec):
| 1.00th=[ 490], 5.00th=[ 519], 10.00th=[ 553], 20.00th=[ 578],
| 30.00th=[ 594], 40.00th=[ 611], 50.00th=[ 627], 60.00th=[ 644],
| 70.00th=[ 676], 80.00th=[ 717], 90.00th=[ 906], 95.00th=[ 47449],
| 99.00th=[261096], 99.50th=[274727], 99.90th=[337642], 99.95th=[337642],
| 99.99th=[337642]
bw ( KiB/s): min=114003, max=1074894, per=98.18%, avg=369010.00, stdev=310212.57, samples=22
iops : min= 27, max= 262, avg=89.59, stdev=75.91, samples=22
lat (usec) : 500=2.34%, 750=81.45%, 1000=7.13%
lat (msec) : 2=0.39%, 10=0.20%, 20=0.10%, 50=3.61%, 100=0.68%
lat (msec) : 250=2.83%, 500=1.27%
cpu : usr=0.04%, sys=20.23%, ctx=10563, majf=0, minf=1060
IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued rwts: total=1024,0,0,0 short=0,0,0,0 dropped=0,0,0,0
latency : target=0, window=0, percentile=100.00%, depth=1
Run status group 0 (all jobs):
READ: bw=367MiB/s (385MB/s), 367MiB/s-367MiB/s (385MB/s-385MB/s), io=4096MiB (4295MB), run=11159-11159msec
$ fio --name=sequential-write -directory=mm --rw=write --refill_buffers --bs=4M --size=4G
sequential-write: (g=0): rw=write, bs=(R) 4096KiB-4096KiB, (W) 4096KiB-4096KiB, (T) 4096KiB-4096KiB, ioengine=psync, iodepth=1 sequential-write: (g=0): rw=write, bs=(R) 4096KiB-4096KiB, (W) 4096KiB-4096KiB, (T) 4096KiB-4096KiB, ioengine=psync, iodepth=1
fio-3.19-68-gf09a77 fio-3.19-68-gf09a77
Starting 1 process Starting 1 process
sequential-write: Laying out IO file (1 file / 4096MiB) Jobs: 1 (f=1): [W(1)][-.-%][w=347MiB/s][w=86 IOPS][eta 00m:00s]
Jobs: 1 (f=1): [W(1)][100.0%][w=59.9MiB/s][w=14 IOPS][eta 00m:00s] sequential-write: (groupid=0, jobs=1): err= 0: pid=35997: Wed Oct 21 11:44:07 2020
sequential-write: (groupid=0, jobs=1): err= 0: pid=63627: Mon May 18 21:31:31 2020 write: IOPS=85, BW=343MiB/s (360MB/s)(1024MiB/2983msec); 0 zone resets
write: IOPS=14, BW=56.2MiB/s (58.9MB/s)(4096MiB/72933msec); 0 zone resets clat (usec): min=2856, max=30851, avg=10274.44, stdev=5417.48
clat (msec): min=3, max=958, avg=70.49, stdev=160.32 lat (usec): min=2857, max=30851, avg=10277.07, stdev=5418.72
lat (msec): min=3, max=958, avg=70.49, stdev=160.32 clat percentiles (usec):
clat percentiles (msec): | 1.00th=[ 2999], 5.00th=[ 3326], 10.00th=[ 3720], 20.00th=[ 5014],
| 1.00th=[ 4], 5.00th=[ 4], 10.00th=[ 4], 20.00th=[ 4], | 30.00th=[ 6718], 40.00th=[ 8029], 50.00th=[ 9634], 60.00th=[10945],
| 30.00th=[ 5], 40.00th=[ 5], 50.00th=[ 5], 60.00th=[ 5], | 70.00th=[12649], 80.00th=[15139], 90.00th=[17433], 95.00th=[20317],
| 70.00th=[ 5], 80.00th=[ 6], 90.00th=[ 220], 95.00th=[ 393], | 99.00th=[27395], 99.50th=[27657], 99.90th=[30802], 99.95th=[30802],
| 99.00th=[ 793], 99.50th=[ 818], 99.90th=[ 944], 99.95th=[ 961], | 99.99th=[30802]
| 99.99th=[ 961] bw ( KiB/s): min=333868, max=374584, per=100.00%, avg=359157.80, stdev=15280.02, samples=5
bw ( KiB/s): min=16286, max=122634, per=100.00%, avg=63938.80, stdev=24402.06, samples=130 iops : min= 81, max= 91, avg=87.40, stdev= 3.78, samples=5
iops : min= 3, max= 29, avg=14.92, stdev= 5.95, samples=130 lat (msec) : 4=12.50%, 10=42.19%, 20=39.84%, 50=5.47%
lat (msec) : 4=23.83%, 10=56.25%, 250=11.82%, 500=4.39%, 750=2.05% cpu : usr=11.10%, sys=31.02%, ctx=1152, majf=0, minf=34
lat (msec) : 1000=1.66%
cpu : usr=1.03%, sys=2.79%, ctx=1963, majf=0, minf=35
IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued rwts: total=0,1024,0,0 short=0,0,0,0 dropped=0,0,0,0 issued rwts: total=0,256,0,0 short=0,0,0,0 dropped=0,0,0,0
latency : target=0, window=0, percentile=100.00%, depth=1 latency : target=0, window=0, percentile=100.00%, depth=1
Run status group 0 (all jobs): Run status group 0 (all jobs):
WRITE: bw=56.2MiB/s (58.9MB/s), 56.2MiB/s-56.2MiB/s (58.9MB/s-58.9MB/s), io=4096MiB (4295MB), run=72933-72933msec WRITE: bw=343MiB/s (360MB/s), 343MiB/s-343MiB/s (360MB/s-360MB/s), io=1024MiB (1074MB), run=2983-2983msec
~$ # restart server and mount again
~$ fio --name=sequential-write -directory=/Users/chris/tmp/mm/buckets/large --rw=read --refill_buffers --bs=4M --size=1G
sequential-write: (g=0): rw=read, bs=(R) 4096KiB-4096KiB, (W) 4096KiB-4096KiB, (T) 4096KiB-4096KiB, ioengine=psync, iodepth=1
fio-3.19-68-gf09a77
Starting 1 process
Jobs: 1 (f=1)
sequential-write: (groupid=0, jobs=1): err= 0: pid=36028: Wed Oct 21 11:44:47 2020
read: IOPS=146, BW=587MiB/s (616MB/s)(1024MiB/1744msec)
clat (usec): min=502, max=97617, avg=6806.27, stdev=19290.63
lat (usec): min=502, max=97617, avg=6806.59, stdev=19290.62
clat percentiles (usec):
| 1.00th=[ 515], 5.00th=[ 545], 10.00th=[ 586], 20.00th=[ 693],
| 30.00th=[ 734], 40.00th=[ 775], 50.00th=[ 824], 60.00th=[ 865],
| 70.00th=[ 906], 80.00th=[ 971], 90.00th=[ 1205], 95.00th=[68682],
| 99.00th=[81265], 99.50th=[81265], 99.90th=[98042], 99.95th=[98042],
| 99.99th=[98042]
bw ( KiB/s): min=496731, max=628270, per=95.07%, avg=571608.33, stdev=67634.94, samples=3
iops : min= 121, max= 153, avg=139.33, stdev=16.50, samples=3
lat (usec) : 750=33.20%, 1000=48.05%
lat (msec) : 2=8.98%, 20=0.78%, 50=1.17%, 100=7.81%
cpu : usr=0.06%, sys=27.08%, ctx=92, majf=0, minf=1060
IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued rwts: total=256,0,0,0 short=0,0,0,0 dropped=0,0,0,0
latency : target=0, window=0, percentile=100.00%, depth=1
Run status group 0 (all jobs):
READ: bw=587MiB/s (616MB/s), 587MiB/s-587MiB/s (616MB/s-616MB/s), io=1024MiB (1074MB), run=1744-1744msec
``` ```
## Common Problems ## ## Common Problems ##