...
______________________________________________
Run Server, N nodes:
...
At last check, 15 nodes are required for all subtests to run, as the rebuild tests will kill servers, and server rejoining server group is not supported today.
orterun --mca mtl ^psm2,ofi -N 1 --hostfile ~/hostlists/daos_server_hostlist --enable-recovery --report-uri ~/scripts/uri.txt daos_server -c 1 &
daos_test
...
:
orterun -x DD_MASK --mca mtl ^psm2,ofi -np 1 --hostfile ~/hostlists/daos_client_hostlist --ompi-server file:~/scripts/uri.txt daos_test -mpcCiAeoRd
CLEAN EVERYTHING, KILL
Relaunch server
Run rebuild tests, N nodes:
orterun --mca mtl ^psm2,ofi -np 1 --hostfile ~/hostlists/daos_client_hostlist --ompi-server file:~/scripts/uri.txt daos_test -r
CLEAN EVERYTHING
Relaunch server
Run rebuild OID allocation tests, N nodes:
orterun -x DD_MASK --mca mtl ^psm2,ofi -np 1 --hostfile ~/hostlists/daos_client_hostlist --ompi-server file:~/scripts/uri.txt daos_test -rO
_______________________________________________
CLEAN EVERYTHING, KILL EVERYTHING
IOR:
_______________________________________________
Set up envirables:
export LD_LIBRARY_PATH=/home/sdwillso/daos_m/opt/ofi/lib/:/home/sdwillso/daos_m/opt/cart/lib/:/home/sdwillso/daos_m/install/lib/:$LD_LIBRARY_PATH
...
Run server:
orterun -x DD_MASK --mca mtl ^psm2,ofi -N 1 --hostfile ~/hostlists/daos_server_hostlist --enable-recovery --report-uri ~/scripts/uri.txt daos_server -c 1 &
Create Pool:
orterun --mca mtl ^psm2,ofi -np 1 --ompi-server file:~/scripts/uri.txt dmg create --size=4G10G
Run IOR:
orterun -np 1 --hostfile ~/hostlists/daos_client_hostlist --mca mtl ^psm2,ofi --ompi-server file:~/scripts/uri.txt ior -v -v W -i 5 -a DAOS -w -o `uuidgen` -b 10g 5g -t 1m -O daospool=$POOL,daosrecordsize=1m,daosstripesize=1m,daosstripecount=1024,daosaios=16,daosobjectclass=LARGE,daosPoolSvc=0$POOL_LEADER,daosepoch=1
_______________________________________________
CLEAN EVERYTHING, KILL EVERYTHING
daosbench:
_______________________________________________
Set environment for single record type implicit aggregation:
-x DAOS_IMPLICIT_PURGE=1 added to orterun server launch command, if desired
Run server
...
:
orterun -x DD_MASK --mca mtl ^psm2,ofi -N 1 --hostfile ~/hostlists/daos_server_hostlist --enable-recovery --report-uri ~/scripts/uri.txt daos_server -c 1 &
Create Pool:
orterun --mca mtl ^psm2,ofi -np 1 --ompi-server file:~/scripts/uri.txt dmg create --size=4G
...
orterun -np 1 --mca mtl ^psm2,ofi ofi --hostfile ~/hostlists/daos_client_hostlist --ompi-server fileserver file:~/scripts/uri.txt daosbench txt daosbench --test=kv-idx-update --testid=1 --svc=0 $POOL_LEADER --dpool=$POOL --container=`uuidgen` --object-class=tiny --aios=32 --indexes=1000000
repeat with: kv-dkey-update, kv-akey-update, kv-dkey-fetch, kv-akey-fetch
_______________________________________________
CLEAN EVERYTHING, KILL EVERYTHING
daos_perf:
_______________________________________________
Set environment for single record type implicit aggregation:
-x DAOS_IMPLICIT_PURGE=1 added to orterun server launch command, if desired
Run server
...
:
orterun -x DD_MASK --mca mtl ^psm2,ofi -N 1 --hostfile ~/hostlists/daos_server_hostlist --enable-recovery --report-uri ~/scripts/uri.txt daos_server -c 1 &
...
1K records:
CREDITS=1 ./daos/daos_m/src/tests/daos_perf.sh daos 200 1000 1K
CREDITS=8 ./daos/daos_m/src/tests/daos_perf.sh daos 200 1000 1K
4k records
CREDITS=1 ./daos/daos_m/src/tests/daos_perf.sh daos 200 1000 4K
...
_______________________________________________
Building
Install autotools latest versions (autoconf, automake, libtool) or use Mohamad's:
export LD_LIBRARY_PATH=/home/mschaara/install/autotools/lib:$LD_LIBRARY_PATH
export PATH=/home/mschaara/install/autotools/bin:$PATH
Setup envirables:
export DAOS_DIR=/home/sdwillso/daos/daos_m/install/
export LD_LIBRARY_PATH=$DAOS_DIR/lib/:$DAOS_DIR/lib/daos_srv:$LD_LIBRARY_PATH
...
export C_INCLUDE_PATH=$DAOS_DIR/include:$C_INCLUDE_PATH
export MPI_LIB=""
Get MPICH with DAOS ADIO branch:
git clone https://github.com/mchaarawi/mpich.git
cd mpich
git checkout daos_adio
...
../configure --prefix=/home/sdwillso/mpich/install/ --enable-fortran=all --enable-romio --enable-cxx --enable-g=all --enable-debuginfo --with-file-system=ufs+daos --with-daos=/home/sdwillso/daos/daos_m/install/ --with-cart=/home/sdwillso/daos/daos_m/opt/cart/
make -j8
...
IMPORTANT NOTE: must keep strict separation between server and client here, ie don't do normal "run server as background process and then run client right after on same node"
Run server:
orterun --enable-recovery -np 1 --hostfile ~/hostlists/daos_server_hostlist --report-uri ~/uri.txt daos_io_server -a/home/sdwillso/tmp/
Set up client envirables:
export PATH=/home/sdwillso/mschaarampich/install/mpich/bin:$PATH
export LD_LIBRARY_PATH=/home/sdwillso/mpich/install/lib:$LD_LIBRARY_PATH
...
export FI_PSM2_DISCONNECT=1
export CRT_CTX_NUM=28
The last two are for PSM2, so you don’t need them if you are using sockets provider.
CRT_ATTACH_INFO_PATH should point to the same thing to what you pass the -a to for daos_server
Build ROMIO test suite:
...
On client:
cd mpich_directory/build/src/mpi/romio/test
make -j8
Run Tests
Set on Server:
export CRT_CTX_NUM=8
Run server:
orterun --mca mtl ^psm2,ofi -N 1 --hostfile ~/hostlists/daos_mpich_hostlist --enable-recovery --report-uri ~/scripts/uri.txt daos_server -a/home/sdwillso/tmp/ -c 1 &
Create a pool with dmg
...
:
mpirun -np 1 dmg create --size=1G
And set env variables ( DAOS_POOL, DAOS_SVCL ) to the pool uuid and svcl rank list accordinglyrespectively.
On Boro I have there is a script that does that in /home/mschaara/create_pool.shThen run
Run:
./run_daos_test tests daos:test_file
Note that the daos: prefix tells mpich to use the daos adio driver. If you don’t have that, it will use the regular posix driver, and you won’t be running with the DAOS driver.
...