Tip of master, commit a88078c2e8ba61b89ad5682be0982e919a200358
All tests run with ofi+psm2, ib0.
daos_test: Run with 8 server (boro-[4-11]), 2 client (boro-[12-13]). Killed servers, cleaned /mnt/daos in between runs listed below.
Tests requiring pool to be created via dmg used 4GB pool. These used boro-12 as client.
mpich tests used boro-4 as server, boro-12 as client, with a 1GB pool.
Separate runs with cleanup in between:
CREDITS=1
[sdwillso@boro-4 ~]$ orterun -x FI_PSM2_DISCONNECT=1 --mca mtl ^psm2,ofi -np 1 -quiet --hostfile /home/sdwillso/scripts/host.cli.1 --ompi-server file:/home/sdwillso/scripts/uri.txt -x DD_SUBSYS= -x DD_MASK= -x D_LOG_FILE=/tmp/daos_perf.log daos_perf -T daos -P 2G -d 1 -a 200 -r 1000 -s 1K -C 1 -t -z Test : DAOS (full stack) Parameters : pool size : 2048 MB credits : 1 (sync I/O for -ve) obj_per_cont : 1 x 1 (procs) dkey_per_obj : 1 akey_per_dkey : 200 recx_per_akey : 1000 value type : single value size : 1024 zero copy : yes overwrite : yes verify fetch : no VOS file : <NULL> 1561f90b: rank 1 became pool service leader 0 Started... update successfully completed: duration : 5.414631 sec bandwith : 36.071 MB/sec rate : 36936.96 IO/sec latency : 27.073 us (nonsense if credits > 1) Duration across processes: MAX duration : 5.414631 sec MIN duration : 5.414631 sec Average duration : 5.414631 sec 1561f90b: rank 1 no longer pool service leader 0 |
CREDITS=8
CREDITS=1
[sdwillso@boro-4 ~]$ orterun -np 1 --hostfile ~/hostlists/daos_client_hostlist --mca mtl ^psm2,ofi --ompi-server file:~/scripts/uri.txt ior -v -W -i 5 -a DAOS -w -o `uuidgen` -b 5g -t 1m -O daospool=9e6d80a4-0709-433e-aea0-502d9782ec32,daosrecordsize=1m,daosstripesize=1m,daosstripecount=1024,daosaios=16,daosobjectclass=LARGE,daosPoolSvc=1,daosepoch=1 IOR-3.0.1: MPI Coordinated Test of Parallel I/O Began: Mon Aug 13 23:19:32 2018 Command line used: ior -v -W -i 5 -a DAOS -w -o cae3b3b4-e0bd-416a-8d31-e6de49df23dc -b 5g -t 1m -O daospool=9e6d80a4-0709-433e-aea0-502d9782ec32,daosrecordsize=1m,daosstripesize=1m,daosstripecount=1024,daosaios=16,daosobjectclass=LARGE,daosPoolSvc=1,daosepoch=1 Machine: Linux boro-12.boro.hpdd.intel.com Start time skew across all tasks: 0.00 sec Test 0 started: Mon Aug 13 23:19:32 2018 Path: /home/sdwillso FS: 3.8 TiB Used FS: 12.7% Inodes: 250.0 Mi Used Inodes: 2.5% Participating tasks: 1 [0] WARNING: USING daosStripeMax CAUSES READS TO RETURN INVALID DATA Summary: api = DAOS test filename = cae3b3b4-e0bd-416a-8d31-e6de49df23dc access = single-shared-file, independent pattern = segmented (1 segment) ordering in a file = sequential offsets ordering inter file= no tasks offsets clients = 1 (1 per node) repetitions = 5 xfersize = 1 MiB blocksize = 5 GiB aggregate filesize = 5 GiB access bw(MiB/s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter ------ --------- ---------- --------- -------- -------- -------- -------- ---- Commencing write performance test: Mon Aug 13 23:19:33 2018 write 5466 5242880 1024.00 0.006637 0.927379 0.002670 0.936711 0 Verifying contents of the file(s) just written. Mon Aug 13 23:19:34 2018 remove - - - - - - 0.002031 0 Commencing write performance test: Mon Aug 13 23:19:37 2018 write 5588 5242880 1024.00 0.001148 0.912741 0.002417 0.916322 1 Verifying contents of the file(s) just written. Mon Aug 13 23:19:38 2018 remove - - - - - - 0.000871 1 Commencing write performance test: Mon Aug 13 23:19:41 2018 write 5573 5242880 1024.00 0.001140 0.915534 0.002083 0.918773 2 Verifying contents of the file(s) just written. Mon Aug 13 23:19:42 2018 remove - - - - - - 0.000875 2 Commencing write performance test: Mon Aug 13 23:19:45 2018 write 5569 5242880 1024.00 0.001136 0.916253 0.001904 0.919309 3 Verifying contents of the file(s) just written. Mon Aug 13 23:19:46 2018 remove - - - - - - 0.000834 3 Commencing write performance test: Mon Aug 13 23:19:50 2018 write 5575 5242880 1024.00 0.001140 0.915108 0.002100 0.918358 4 Verifying contents of the file(s) just written. Mon Aug 13 23:19:50 2018 remove - - - - - - 0.000826 4 Max Write: 5587.55 MiB/sec (5858.98 MB/sec) Summary of all tests: Operation Max(MiB) Min(MiB) Mean(MiB) StdDev Mean(s) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt blksiz xsize aggsize API RefNum write 5587.55 5465.93 5554.14 44.53 0.92189 0 1 1 5 0 0 1 0 0 1 5368709120 1048576 5368709120 DAOS 0 Finished: Mon Aug 13 23:19:56 2018 |
Time: 18.896496 seconds (52919.864571 ops per second)
[sdwillso@boro-4 ~]$ orterun -np 1 -x FI_PSM2_DISCONNECT=1 --mca mtl ^psm2,ofi --hostfile ~/hostlists/daos_client_hostlist --ompi-server file:~/scripts/uri.txt daosbench --test=kv-idx-update --testid=1 --svc=0 --dpool=2e0fbf59-f1e5-4695-a86f-595fe29656fe --container=`uuidgen` --object-class=large --aios=32 --indexes=1000000 ================================ DAOSBENCH (KV) Started at Wed Aug 15 22:01:48 2018 ================================= =============================== Test Setup --------------- Test: kv-idx-update DAOS pool :2e0fbf59-f1e5-4695-a86f-595fe29656fe DAOS container :9e80da19-23d8-46b8-9e8d-6fcadf6f573f Value buffer size: 64 Number of processes: 1 Number of indexes/process: 1000000 Number of asynchronous I/O: 32 =============================== kv-idx-update Time: 18.896496 seconds (52919.864571 ops per second) Ended at Wed Aug 15 22:02:09 2018 |
Time: 0.008656 seconds (11552.226834 ops per second)
[sdwillso@boro-4 ~]$ orterun -np 1 -x FI_PSM2_DISCONNECT=1 --mca mtl ^psm2,ofi --hostfile ~/hostlists/daos_client_hostlist --ompi-server file:~/scripts/uri.txt daosbench --test=kv-dkey-update --testid=1 --svc=1 --dpool=dfa53c85-55ea-421a-902f-c4f2e41e74fa --container=`uuidgen` --object-class=tiny --aios=32 --indexes=1000000 ================================ DAOSBENCH (KV) Started at Wed Aug 15 21:35:38 2018 ================================= =============================== Test Setup --------------- Test: kv-dkey-update DAOS pool :dfa53c85-55ea-421a-902f-c4f2e41e74fa DAOS container :7454ae93-6783-4996-affc-83458ffd3017 Value buffer size: 64 Number of processes: 1 Number of keys/process: 100 Number of asynchronous I/O: 32 =============================== kv-dkey-update Time: 0.008656 seconds (11552.226834 ops per second) Ended at Wed Aug 15 21:35:39 2018 |
Time: 0.004028 seconds (24828.409790 ops per second)
[sdwillso@boro-4 ~]$ orterun -np 1 -x FI_PSM2_DISCONNECT=1 --mca mtl ^psm2,ofi --hostfile ~/hostlists/daos_client_hostlist --ompi-server file:~/scripts/uri.txt daosbench --test=kv-akey-update --testid=1 --svc=1 --dpool=6032ac76-ec42-4306-990a-c5f41ececaca --container=`uuidgen` --object-class=tiny --aios=32 --indexes=1000000 ================================ DAOSBENCH (KV) Started at Wed Aug 15 21:37:25 2018 ================================= =============================== Test Setup --------------- Test: kv-akey-update DAOS pool :6032ac76-ec42-4306-990a-c5f41ececaca DAOS container :79e51a0a-5fbc-4824-991e-2153298054c8 Value buffer size: 64 Number of processes: 1 Number of keys/process: 100 Number of asynchronous I/O: 32 =============================== kv-akey-update Time: 0.004028 seconds (24828.409790 ops per second) Ended at Wed Aug 15 21:37:26 2018 |
Time: 0.000558 seconds (179346.151816 ops per second)
[sdwillso@boro-4 ~]$ orterun -np 1 -x FI_PSM2_DISCONNECT=1 --mca mtl ^psm2,ofi --hostfile ~/hostlists/daos_client_hostlist --ompi-server file:~/scripts/uri.txt daosbench --test=kv-dkey-fetch --testid=1 --svc=1 --dpool=f8971edf-863c-4230-b8c6-84c470777754 --container=`uuidgen` --object-class=tiny --aios=32 --indexes=1000000 ================================ DAOSBENCH (KV) Started at Wed Aug 15 21:38:55 2018 ================================= =============================== Test Setup --------------- Test: kv-dkey-fetch DAOS pool :f8971edf-863c-4230-b8c6-84c470777754 DAOS container :25178851-633b-4ca8-b0f4-dea984c972a0 Value buffer size: 64 Number of processes: 1 Number of keys/process: 100 Number of asynchronous I/O: 32 =============================== kv-dkey-fetch Time: 0.000558 seconds (179346.151816 ops per second) Ended at Wed Aug 15 21:38:57 2018 |
Time: 0.001551 seconds (64472.580196 ops per second)
[sdwillso@boro-4 ~]$ orterun -np 1 -x FI_PSM2_DISCONNECT=1 --mca mtl ^psm2,ofi --hostfile ~/hostlists/daos_client_hostlist --ompi-server file:~/scripts/uri.txt daosbench --test=kv-akey-fetch --testid=1 --svc=1 --dpool=94825fa8-4d61-42b3-b0a4-062e7d7815d4 --container=`uuidgen` --object-class=tiny --aios=32 --indexes=1000000 ================================ DAOSBENCH (KV) Started at Wed Aug 15 21:40:21 2018 ================================= =============================== Test Setup --------------- Test: kv-akey-fetch DAOS pool :94825fa8-4d61-42b3-b0a4-062e7d7815d4 DAOS container :51da27a6-1b3a-4627-84c4-c6c04284bd3c Value buffer size: 64 Number of processes: 1 Number of keys/process: 100 Number of asynchronous I/O: 32 =============================== kv-akey-fetch Time: 0.001551 seconds (64472.580196 ops per second) Ended at Wed Aug 15 21:40:22 2018 |