Write Type: fillseq Input Data Size: 1GB Cache Size: 0MB Written Data: 1669.75 MB fillseq: 321.309 micros/op 3112 ops/sec; 3.1 MB/s Running Time: 6 minutes Write Type: fillsync Input Data Size: 1GB Cache Size: 0MB Written Data: 38724.7 MB fillsync: 604.234 micros/op 1654 ops/sec; 1.6 MB/s (1048 ops) Running Time: 10:56 minutes Write Type: fillrandomsync Input Data Size: 1GB Cache Size: 0MB Written Data: 38995.1 MB fillrandomsync: 608.505 micros/op 1643 ops/sec; 1.6 MB/s Running Time: 11:01 minutes
sudo ./run_benchmark.sh -f -n 1048576 -b fillsync -c 134217728 53.36s user 80.16s system 59% cpu 3:43.15 total fillsync : 190.635 micros/op 5245 ops/sec; 5.2 MB/s (1048 ops) /tmp/kk 1.04785 MB read 7861.96 MB written sudo ./run_benchmark.sh -f -n 1048576 -b fillrandomsync -c 134217728 53.18s user 80.95s system 59% cpu 3:47.07 total fillrandomsync : 190.659 micros/op 5244 ops/sec; 5.2 MB/s /tmp/kk 1.04785 MB read 7860.55 MB written
Write Type: fillseq Input Data Size: 4GB Cache Size: 0MB Written Data: 5367.59 fillseq : 1918.276 micros/op 521 ops/sec; 0.5 MB/s Running Time: 2:14:27 Write Type: fillsync Input Data Size: 4GB Cache Size: 0MB Written Data: 161780 MB fillsync : 647.437 micros/op 1544 ops/sec; 1.5 MB/s (4194 ops) Running Time: 45:44 minutes Write Type: fillrandomsync Input Data Size: 4GB Cache Size: 0MB Written Data: 161892 MB fillrandomsync : 646.120 micros/op 1547 ops/sec; 1.5 MB/s Running Time: 45:36 minutes
sudo ./run_benchmark.sh -f -n 4194304 -b fillseq -c 536870912 41.14s user 14.95s system 89% cpu 1:02.98 totaltotal fillseq : 7.497 micros/op 133391 ops/sec; 132.3 MB/s /tmp/kk 1.04004 MB read 5898.29 MB written sudo ./run_benchmark.sh -f -n 4194304 -b fillsync -c 536870912 247.12s user 333.16s system 68% cpu 14:10.15 total fillsync : 197.111 micros/op 5073 ops/sec; 5.0 MB/s (4194 ops) /tmp/kk 1.04785 MB read 31754.1 MB written sudo ./run_benchmark.sh -f -n 4194304 -b fillrandomsync -c 536870912 245.78s user 333.24s system 67% cpu 14:23.00 total fillrandomsync : 196.207 micros/op 5096 ops/sec; 5.1 MB/s /tmp/kk 1.04785 MB read 31749.1 MB written
The following values were obtained using iostat while running a workload defined by the string “kv_open:kv_put:kv_get:kv_close”. I tried running the puts separated from the gets but it failes b/c the keys that are putted are generated randomly and then are not found by get. There is an options to generate the keys sequentially avoiding the problem, but when enabled, this causes the put to failed returning a NVM_ERR_OBJECT_EXISTS error.
nvmkv-1.0.0.12_kv_open:kv_put:kv_get:kv_close_d1024mb_c0mb_t1_iostat 1523.58 MB read 1536.01 MB written nvmkv-1.0.0.12-cache_kv_open:kv_put:kv_get:kv_close_d1024mb_c128mb_t1_iostat 1526.69 MB read 1536.01 MB written nvmkv-1.0.0.12-cache_kv_open:kv_put:kv_get:kv_close_d4096mb_c512mb_t1_iostat 6135.87 MB read 6143.98 MB written nvmkv-1.0.0.12-cache_kv_open:kv_put:kv_get:kv_close_d16384mb_c2048mb_t1_iostat 24574.6 MB read 24575.9 MB written
nvmkv-1.0.0.12-cache_kv_open:kv_put:kv_get:kv_close_d1024mb_c0mb_t1 KV PUT - time delta = 65873928, Avg latency = 62, IOPS = 15918 KV PUT Data throughput 15.54MB/s Using a 1-thread-pool to test kv_get KV GET - time delta = 114618446, Avg latency = 109, IOPS = 9148 KV GET Data throughput 89.34MB/s nvmkv-1.0.0.12-cache_kv_open:kv_put:kv_get:kv_close_d4096mb_c0mb_t1 KV PUT - time delta = 262166962, Avg latency = 62, IOPS = 15999 KV PUT Data throughput 0.00MB/s Using a 1-thread-pool to test kv_get KV GET - time delta = 457883503, Avg latency = 109, IOPS = 9160 KV GET Data throughput 0.00MB/s nvmkv-1.0.0.12-cache_kv_open:kv_put:kv_get:kv_close_d16384mb_c0mb_t1 KV PUT - time delta = 1077625943, Avg latency = 64, IOPS = 15569 KV PUT Data throughput 0.00MB/s Using a 1-thread-pool to test kv_get KV GET - time delta = 1834484677, Avg latency = 109, IOPS = 9145 KV GET Data throughput 0.00MB/s
nvmkv-1.0.0.12-cache_kv_open:kv_put:kv_get:kv_close_d1024mb_c128mb_t1 KV PUT - time delta = 76605511, Avg latency = 73, IOPS = 13688 KV PUT Data throughput 13.37MB/s Using a 1-thread-pool to test kv_get KV GET - time delta = 125195104, Avg latency = 119, IOPS = 8376 KV GET Data throughput 81.79MB/s nvmkv-1.0.0.12-cache_kv_open:kv_put:kv_get:kv_close_d4096mb_c512mb_t1 KV PUT - time delta = 308835896, Avg latency = 73, IOPS = 13581 KV PUT Data throughput 0.00MB/s Using a 1-thread-pool to test kv_get KV GET - time delta = 508229441, Avg latency = 121, IOPS = 8253 KV GET Data throughput 0.00MB/s nvmkv-1.0.0.12-cache_kv_open:kv_put:kv_get:kv_close_d16384mb_c2048mb_t1 KV PUT - time delta = 1276636342, Avg latency = 76, IOPS = 13142 KV PUT Data throughput 0.00MB/s Using a 1-thread-pool to test kv_get KV GET - time delta = 2055414591, Avg latency = 122, IOPS = 8162 KV GET Data throughput 0.00MB/s
These read amplification numbers make no sense. We should read at least 1GB of data, which is what's being written.
leveldb.git_fillseq_c0mb_d1024mb_xfs_iostat 1.05664 MB read 2707.23 MB written leveldb.git_fillseq_c128mb_d1024mb_xfs_iostat 1.05664 MB read 2682.41 MB written leveldb.git_fillrandom_c0mb_d1024mb_xfs_iostat 1.05664 MB read 34823.3 MB written leveldb.git_fillrandom_c128mb_d1024mb_xfs_iostat 1.05664 MB read 34391.7 MB written leveldb.git_fillrandomsync_c0mb_d1024mb_xfs_iostat 1.06543 MB read 43603.7 MB written leveldb.git_fillrandomsync_c128mb_d1024mb_xfs_iostat 1.08301 MB read 43348.5 MB written leveldb.git_fillsync_c0mb_d1024mb_xfs_iostat 1.05664 MB read 2650.86 MB written leveldb.git_fillsync_c128mb_d1024mb_xfs_iostat 1.02734 MB read 386.876 MB written The following number are not valid leveldb.git_readseq_c0mb_d1024mb_xfs_iostat 1.04004 MB read 2013.05 MB written leveldb.git_readseq_c129mb_d1024mb_xfs_iostat 5.84961 MB read 2706.62 MB written leveldb.git_readrandom_c0mb_d1024mb_xfs_iostat 1.04004 MB read 2013.13 MB written leveldb.git_readrandom_c128mb_d1024mb_xfs_iostat 5.84961 MB read 2706.7 MB written
Notice that the read operations did not find the key-value paris. This may be the reason for the meaningless read amplification numbers.
leveldb.git_fillseq_c0mb_d1024mb_xfs_iostat fillseq : 163.081 micros/op; 6.1 MB/s leveldb.git_fillseq_c128mb_d1024mb_xfs_iostat fillseq : 163.393 micros/op; 6.1 MB/s leveldb.git_fillrandom_c0mb_d1024mb_xfs fillrandom : 547.799 micros/op; 1.8 MB/s leveldb.git_fillrandom_c128mb_d1024mb_xfs fillrandom : 541.217 micros/op; 1.8 MB/s leveldb.git_fillrandomsync_c0mb_d1024mb_xfs fillrandomsync : 645.497 micros/op; 1.5 MB/s leveldb.git_fillrandomsync_c128mb_d1024mb_xfs fillrandomsync : 650.001 micros/op; 1.5 MB/s leveldb.git_fillsync_c0mb_d1024mb_xfs fillsync : 156.158 micros/op; 6.4 MB/s (1048 ops) leveldb.git_fillsync_c128mb_d1024mb_xfs fillsync : 159.961 micros/op; 6.2 MB/s (1048 ops) leveldb.git_readseq_c0mb_d1024mb_xfs readseq : 9.147 micros/op; 108 MB/s leveldb.git_readseq_c128mb_d1024mb_xfs readseq : 1.875 micros/op; 529 MB/s leveldb.git_readrandom_c0mb_d1024mb_xfs readrandom : 31.619 micros/op; (1048576 of 1048576 found) leveldb.git_readrandom_c128mb_d1024mb_xfs readrandom : 8.423 micros/op; (1048576 of 1048576 found)
leveldb.git_fillrandom_c0mb_d4096mb_xfs_iostat 1.05664 MB read 141366 MB written leveldb.git_fillrandom_c536870912mb_d4096mb_xfs_iostat 1.04785 MB read 6772.15 MB written leveldb.git_fillrandomsync_c0mb_d4096mb_xfs_iostat 1.07422 MB read 179847 MB written leveldb.git_fillrandomsync_c536870912mb_d4096mb_xfs_iostat 1.06543 MB read 30967.6 MB written leveldb.git_fillseq_c0mb_d4096mb_xfs_iostat 1.05664 MB read 9644.59 MB written leveldb.git_fillseq_c536870912mb_d4096mb_xfs_iostat 1.04004 MB read 6891.19 MB written leveldb.git_fillsync_c0mb_d4096mb_xfs_iostat 1.06734 MB read 95611.8 MB written leveldb.git_fillsync_c536870912mb_d4096mb_xfs_iostat 1.02734 MB read 386.876 MB written The following numbers are not valid leveldb.git_readrandom_c0mb_d4096mb_xfs_iostat 1.04004 MB read 6891.3 MB written leveldb.git_readrandom_c536870912mb_d4096mb_xfs_iostat 1.04004 MB read 6891.34 MB written leveldb.git_readseq_c0mb_d4096mb_xfs_iostat 1.04004 MB read 6891.2 MB written leveldb.git_readseq_c536870912mb_d4096mb_xfs_iostat 1.04004 MB read 6891.24 MB written
leveldb.git_fillrandom_c0mb_d4096mb_xfs fillrandom : 561.553 micros/op; 1.8 MB/s leveldb.git_fillrandom_c536870912mb_d4096mb_xfs fillrandom : 9.802 micros/op; 101.2 MB/s leveldb.git_fillrandomsync_c0mb_d4096mb_xfs fillrandomsync : 674.941 micros/op; 1.5 MB/s leveldb.git_fillrandomsync_c536870912mb_d4096mb_xfs fillrandomsync : 146.057 micros/op; 6.8 MB/s leveldb.git_fillseq_c0mb_d4096mb_xfs fillseq : 739.931 micros/op; 1.3 MB/s leveldb.git_fillseq_c536870912mb_d4096mb_xfs fillseq : 6.927 micros/op; 143.2 MB/s leveldb.git_fillsync_c0mb_d4096mb_xfs fillsync : 742.362 micros/op; 1.3 MB/s (4194 ops) leveldb.git_fillsync_c536870912mb_d4096mb_xfs fillsync : 120.186 micros/op; 8.3 MB/s (4194 ops) leveldb.git_readrandom_c0mb_d4096mb_xfs readrandom : 101.547 micros/op; (4194304 of 4194304 found) leveldb.git_readrandom_c536870912mb_d4096mb_xfs readrandom : 16.548 micros/op; (4194304 of 4194304 found) leveldb.git_readseq_c0mb_d4096mb_xfs readseq : 18.836 micros/op; 52.6 MB/s leveldb.git_readseq_c512mb_d4096mb_xfs readseq : 1.831 micros/op; 541.7 MB/s