5-10-18
Former user (Deleted)
Owned by Former user (Deleted)
May 22, 2018
11 min read
Loading data...
Test Configuration
Run with ofi+psm2, ib0.
daos_test: Run with 8 server (boro-[3-10]), 2 client (boro-[11-12]). Killed servers, cleaned /mnt/daos in between runs listed below.
Tests requiring pool to be created via dmg used 4GB pool. These used boro-11 as client.
Test Results
daos_test
Separate runs with cleanup in between:
- -mpcCiAeoRd - PASS
- -r - PASS
daosperf
1K Records
CREDITS=1
Expand source
[sdwillso@boro-11 ~]$ CREDITS=1 ./daos_m/src/tests/daos_perf.sh daos 200 1000 1K + /home/sdwillso/daos_m/opt/ompi/bin/orterun -quiet --hostfile /home/sdwillso/scripts/host.cli.1 --ompi-server file:/home/sdwillso/scripts/uri.txt -x DD_SUBSYS= -x DD_MASK= -x D_LOG_FILE=/tmp/daos_perf.log /home/sdwillso/daos_m/install/bin/daos_perf -T daos -P 2G -d 1 -a 200 -r 1000 -s 1K -C 1 -t -z Test : DAOS (full stack) Parameters : pool size : 2048 MB credits : 1 (sync I/O for -ve) obj_per_cont : 1 x 8 (procs) dkey_per_obj : 1 akey_per_dkey : 200 recx_per_akey : 1000 value type : single value size : 1024 zero copy : yes overwrite : yes VOS file : <NULL> Started... update successfully completed: duration : 12.927818 sec bandwith : 120.863 MB/sec rate : 123764.12 IO/sec latency : 8.080 us (nonsense if credits > 1) Duration across processes: MAX duration : 12.924732 sec MIN duration : 5.433614 sec Average duration : 8.982438 sec
CREDITS=8
Expand source
[sdwillso@boro-11 ~]$ CREDITS=8 ./daos_m/src/tests/daos_perf.sh daos 200 1000 1K + /home/sdwillso/daos_m/opt/ompi/bin/orterun -quiet --hostfile /home/sdwillso/scripts/host.cli.1 --ompi-server file:/home/sdwillso/scripts/uri.txt -x DD_SUBSYS= -x DD_MASK= -x D_LOG_FILE=/tmp/daos_perf.log /home/sdwillso/daos_m/install/bin/daos_perf -T daos -P 2G -d 1 -a 200 -r 1000 -s 1K -C 8 -t -z Test : DAOS (full stack) Parameters : pool size : 2048 MB credits : 8 (sync I/O for -ve) obj_per_cont : 1 x 8 (procs) dkey_per_obj : 1 akey_per_dkey : 200 recx_per_akey : 1000 value type : single value size : 1024 zero copy : yes overwrite : yes VOS file : <NULL> Started... update successfully completed: duration : 7.282236 sec bandwith : 214.563 MB/sec rate : 219712.73 IO/sec latency : 4.551 us (nonsense if credits > 1) Duration across processes: MAX duration : 7.280755 sec MIN duration : 3.605414 sec Average duration : 5.563875 sec
4K Records
CREDITS=1
Expand source
[sdwillso@boro-11 ~]$ CREDITS=1 ./daos_m/src/tests/daos_perf.sh daos 200 1000 4K + /home/sdwillso/daos_m/opt/ompi/bin/orterun -quiet --hostfile /home/sdwillso/scripts/host.cli.1 --ompi-server file:/home/sdwillso/scripts/uri.txt -x DD_SUBSYS= -x DD_MASK= -x D_LOG_FILE=/tmp/daos_perf.log /home/sdwillso/daos_m/install/bin/daos_perf -T daos -P 2G -d 1 -a 200 -r 1000 -s 4K -C 1 -t -z Test : DAOS (full stack) Parameters : pool size : 2048 MB credits : 1 (sync I/O for -ve) obj_per_cont : 1 x 8 (procs) dkey_per_obj : 1 akey_per_dkey : 200 recx_per_akey : 1000 value type : single value size : 4096 zero copy : yes overwrite : yes VOS file : <NULL> Started... update successfully completed: duration : 14.677129 sec bandwith : 425.833 MB/sec rate : 109013.15 IO/sec latency : 9.173 us (nonsense if credits > 1) Duration across processes: MAX duration : 14.676090 sec MIN duration : 6.163768 sec Average duration : 9.740256 sec
IOR
Expand source
[sdwillso@boro-11 ~]$ orterun -np 1 --mca mtl ^psm2,ofi --ompi-server file:~/scripts/uri.txt ior -v -i 5 -a DAOS -w -o `uuidgen` -b 10g -t 1m -O daospool=8cea4438-41a4-4a86-944c-91eededb7f32,daosrecordsize=1m,daosstripesize=1m,daosstripecount=1024,daosaios=16,daosobjectclass=LARGE,daosPoolSvc=1,daosepoch=1 IOR-3.0.1: MPI Coordinated Test of Parallel I/O Began: Wed May 9 21:31:11 2018 Command line used: ior -v -i 5 -a DAOS -w -o 22ca6af0-80b5-42eb-8014-1d0f0574f10b -b 10g -t 1m -O daospool=8cea4438-41a4-4a86-944c-91eededb7f32,daosrecordsize=1m,daosstripesize=1m,daosstripecount=1024,daosaios=16,daosobjectclass=LARGE,daosPoolSvc=1,daosepoch=1 Machine: Linux boro-11.boro.hpdd.intel.com Start time skew across all tasks: 0.00 sec Test 0 started: Wed May 9 21:31:11 2018 Path: /home/sdwillso FS: 3.8 TiB Used FS: 9.1% Inodes: 250.0 Mi Used Inodes: 1.8% Participating tasks: 1 [0] WARNING: USING daosStripeMax CAUSES READS TO RETURN INVALID DATA Summary: api = DAOS test filename = 22ca6af0-80b5-42eb-8014-1d0f0574f10b access = single-shared-file, independent pattern = segmented (1 segment) ordering in a file = sequential offsets ordering inter file= no tasks offsets clients = 1 (1 per node) repetitions = 5 xfersize = 1 MiB blocksize = 10 GiB aggregate filesize = 10 GiB access bw(MiB/s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter ------ --------- ---------- --------- -------- -------- -------- -------- ---- Commencing write performance test: Wed May 9 21:31:11 2018 write 5873 10485760 1024.00 0.000820 1.74 0.002990 1.74 0 remove - - - - - - 0.004431 0 Commencing write performance test: Wed May 9 21:31:12 2018 write 5787 10485760 1024.00 0.000641 1.77 0.003038 1.77 1 remove - - - - - - 0.003924 1 Commencing write performance test: Wed May 9 21:31:14 2018 write 5819 10485760 1024.00 0.000631 1.76 0.003006 1.76 2 remove - - - - - - 0.003853 2 Commencing write performance test: Wed May 9 21:31:16 2018 write 5784 10485760 1024.00 0.000653 1.77 0.002409 1.77 3 remove - - - - - - 0.003700 3 Commencing write performance test: Wed May 9 21:31:18 2018 write 5852 10485760 1024.00 0.000623 1.75 0.002210 1.75 4 remove - - - - - - 0.003672 4 Max Write: 5872.96 MiB/sec (6158.24 MB/sec) Summary of all tests: Operation Max(MiB) Min(MiB) Mean(MiB) StdDev Mean(s) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt blksiz xsize aggsize API RefNum write 5872.96 5784.41 5823.06 35.00 1.75859 0 1 1 5 0 0 1 0 0 1 10737418240 1048576 10737418240 DAOS 0
daos_bench
kv-idx-update
Time: 61.649248 seconds (16220.797915 ops per second)
Expand source
[sdwillso@boro-11 ~]$ orterun -np 1 --mca mtl ^psm2,ofi --ompi-server file:~/scripts/uri.txt daosbench --test=kv-idx-update --testid=1 --svc=1 --dpool=63566c58-07c9-475d-b4eb-5e48183ffa96 --container=`uuidgen` --object-class=tiny --aios=32 --indexes=1000000 ================================ DAOSBENCH (KV) Started at Wed May 9 21:35:16 2018 ================================= =============================== Test Setup --------------- Test: kv-idx-update DAOS pool :63566c58-07c9-475d-b4eb-5e48183ffa96 DAOS container :48f4af67-3e70-4f77-a760-6a251910b6aa Value buffer size: 64 Number of processes: 1 Number of indexes/process: 1000000 Number of asynchronous I/O: 32 =============================== kv-idx-update Time: 61.649248 seconds (16220.797915 ops per second)
kv-dkey-update
Time: 0.011496 seconds (8698.559066 ops per second)
Expand source
[sdwillso@boro-11 ~]$ orterun -np 1 --mca mtl ^psm2,ofi --ompi-server file:~/scripts/uri.txt daosbench --test=kv-dkey-update --testid=1 --svc=1 --dpool=36b19077-a5c1-4b51-9518-0d4a422bc373 --container=`uuidgen` --object-class=tiny --aios=32 --indexes=1000000 ================================ DAOSBENCH (KV) Started at Wed May 9 21:43:32 2018 ================================= =============================== Test Setup --------------- Test: kv-dkey-update DAOS pool :36b19077-a5c1-4b51-9518-0d4a422bc373 DAOS container :a44be6b6-47cb-4109-8a4a-3f303db78c0e Value buffer size: 64 Number of processes: 1 Number of keys/process: 100 Number of asynchronous I/O: 32 =============================== kv-dkey-update Time: 0.011496 seconds (8698.559066 ops per second)
kv-akey-update
Time: 0.010799 seconds (9259.803600 ops per second)
Expand source
[sdwillso@boro-11 ~]$ orterun -np 1 --mca mtl ^psm2,ofi --ompi-server file:~/scripts/uri.txt daosbench --test=kv-akey-update --testid=1 --svc=1 --dpool=e51969c5-9939-40b5-aba7-91c98eb713c3 --container=`uuidgen` --object-class=tiny --aios=32 --indexes=1000000 ================================ DAOSBENCH (KV) Started at Thu May 10 00:01:10 2018 ================================= =============================== Test Setup --------------- Test: kv-akey-update DAOS pool :e51969c5-9939-40b5-aba7-91c98eb713c3 DAOS container :fb928db7-b107-4dd3-b651-c642bc32ec5c Value buffer size: 64 Number of processes: 1 Number of keys/process: 100 Number of asynchronous I/O: 32 =============================== kv-akey-update Time: 0.010799 seconds (9259.803600 ops per second)
kv-dkey-fetch
Time: 0.006289 seconds (15900.950636 ops per second)
Expand source
[sdwillso@boro-11 ~]$ orterun -np 1 --mca mtl ^psm2,ofi --ompi-server file:~/scripts/uri.txt daosbench --test=kv-dkey-fetch --testid=1 --svc=1 --dpool=db7422a4-48a7-4808-8969-1f9c5eab47a1 --container=`uuidgen` --object-class=tiny --aios=32 --indexes=1000000 ================================ DAOSBENCH (KV) Started at Thu May 10 00:04:48 2018 ================================= =============================== Test Setup --------------- Test: kv-dkey-fetch DAOS pool :db7422a4-48a7-4808-8969-1f9c5eab47a1 DAOS container :605cb81f-fb14-42bc-b783-48e94514aa4d Value buffer size: 64 Number of processes: 1 Number of keys/process: 100 Number of asynchronous I/O: 32 =============================== kv-dkey-fetch Time: 0.006289 seconds (15900.950636 ops per second)
kv-akey-fetch
Time: 0.007472 seconds (13382.482667 ops per second)
Expand source
[sdwillso@boro-11 ~]$ orterun -np 1 --mca mtl ^psm2,ofi --ompi-server file:~/scripts/uri.txt daosbench --test=kv-akey-fetch --testid=1 --svc=1 --dpool=e7ca342e-856f-480d-b1a3-36fe2c558d4a --container=`uuidgen` --object-class=tiny --aios=32 --indexes=1000000 ================================ DAOSBENCH (KV) Started at Thu May 10 00:10:29 2018 ================================= =============================== Test Setup --------------- Test: kv-akey-fetch DAOS pool :e7ca342e-856f-480d-b1a3-36fe2c558d4a DAOS container :b0e0da88-99da-4664-b48d-83568c31b307 Value buffer size: 64 Number of processes: 1 Number of keys/process: 100 Number of asynchronous I/O: 32 =============================== kv-akey-fetch Time: 0.007472 seconds (13382.482667 ops per second)
mpich tests
Results: No Errors on all tests
Expand source
[sdwillso@boro-11 test]$ ./run_daos_tests daos:test_file **** Testing I/O functions **** **** Testing simple.c **** POOL UUID = 565212e9-28e1-42b5-ae4d-c4efa1d8db8f SVC LIST = 0 No Errors **** Testing async.c **** POOL UUID = 565212e9-28e1-42b5-ae4d-c4efa1d8db8f SVC LIST = 0 No Errors **** Testing async-multiple.c **** POOL UUID = 565212e9-28e1-42b5-ae4d-c4efa1d8db8f SVC LIST = 0 No Errors **** Testing coll_test.c **** POOL UUID = 565212e9-28e1-42b5-ae4d-c4efa1d8db8f SVC LIST = 0 No Errors **** Testing excl.c **** POOL UUID = 565212e9-28e1-42b5-ae4d-c4efa1d8db8f SVC LIST = 0 ../../../../src/mpi/romio/adio/ad_daos/ad_daos_open.c:281 ADIOI_DAOS_Open() - Array exists (EXCL mode) (-1004) No Errors **** Testing file_info.c **** POOL UUID = 565212e9-28e1-42b5-ae4d-c4efa1d8db8f SVC LIST = 0 No Errors **** Testing i_noncontig.c **** POOL UUID = 565212e9-28e1-42b5-ae4d-c4efa1d8db8f SVC LIST = 0 No Errors **** Testing noncontig.c **** POOL UUID = 565212e9-28e1-42b5-ae4d-c4efa1d8db8f SVC LIST = 0 No Errors **** Testing noncontig_coll.c **** POOL UUID = 565212e9-28e1-42b5-ae4d-c4efa1d8db8f SVC LIST = 0 No Errors **** Testing noncontig_coll2.c **** POOL UUID = 565212e9-28e1-42b5-ae4d-c4efa1d8db8f SVC LIST = 0 No Errors **** Testing aggregation1.c **** POOL UUID = 565212e9-28e1-42b5-ae4d-c4efa1d8db8f SVC LIST = 0 No Errors **** Testing aggregation2.c **** POOL UUID = 565212e9-28e1-42b5-ae4d-c4efa1d8db8f SVC LIST = 0 No Errors **** Testing hindexed **** POOL UUID = 565212e9-28e1-42b5-ae4d-c4efa1d8db8f SVC LIST = 0 ------------------------------------------------------- [ 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 ] [ 0] 0 1 2 3 4 5 D E F G H I [ 1] [ 2] 6 7 8 9 : ; J K L M N O [ 3] [ 4] [ 5] X Y Z [ \ ] l m n o p q [ 6] [ 7] ^ _ ` a b c r s t u v w [ 8] [ 9] [10] 0 1 2 3 4 5 D E F G H I [11] [12] 6 7 8 9 : ; J K L M N O [13] [14] [15] X Y Z [ \ ] l m n o p q [16] [17] ^ _ ` a b c r s t u v w [18] [19] No Errors **** Testing split_coll.c **** POOL UUID = 565212e9-28e1-42b5-ae4d-c4efa1d8db8f SVC LIST = 0 No Errors **** Testing psimple.c **** POOL UUID = 565212e9-28e1-42b5-ae4d-c4efa1d8db8f SVC LIST = 0 No Errors **** Testing error.c **** POOL UUID = 565212e9-28e1-42b5-ae4d-c4efa1d8db8f SVC LIST = 0 No Errors **** Testing status.c **** POOL UUID = 565212e9-28e1-42b5-ae4d-c4efa1d8db8f SVC LIST = 0 No Errors **** Testing types_with_zeros **** POOL UUID = 565212e9-28e1-42b5-ae4d-c4efa1d8db8f SVC LIST = 0 No Errors **** Testing darray_read **** POOL UUID = 565212e9-28e1-42b5-ae4d-c4efa1d8db8f SVC LIST = 0 No Errors **** Testing fcoll_test.f **** POOL UUID = 565212e9-28e1-42b5-ae4d-c4efa1d8db8f SVC LIST = 0 No Errors **** Testing pfcoll_test.f **** POOL UUID = 565212e9-28e1-42b5-ae4d-c4efa1d8db8f SVC LIST = 0 No Errors