...
Environment variables (If set any):
| Testing Area | Test | Test Priority (1- HIGH, 2 - LOW) | Number of Servers | Number of Clients | Input Parameter | Expected Result | Observed Result | Defect | Notes | Expected SU's (1 node * 1 hour = 1 SU) | |
|---|---|---|---|---|---|---|---|---|---|---|---|
| Server YAML config options | To verify the test cases from below section with specific server config options in YAML file | 1 | target = [16] nr_xs_helpers = [1] CRT_CTX_SHARE_ADDR=[0, 1] | No sever crash, Performance increase linearly | No need individual test but below test can be used this configuration | ||||||
| Performance | No | ||||||||||
Replica Run IOR and collect BW Run IOR small size and collect IOPS | 1 | 1, 8, 32, 128 128 | 1, 16, 96, 256 740 | protocol : daos Transfer Size: 256B 4K 128K 512K 1M (Do we need non standard size also be covered?) Block Size: 64M (Depend upon no. of process as file size will increase because of it) FPP and SSF | single server got ~12GB Read/write so it should scale linearly. | 1406 Nodes taking ~30 min | 703 | ||||
Replica 2 Way Run IOR and collect BW Run IOR small size and collect IOPS | 1 | 8, 32, 128 | 16, 96, 740 | ||||||||
Replica 3 Way Run IOR and collect BW Run IOR small size and collect IOPS | 1 | 8, 32, 128 | 16, 96, 740 | ||||||||
Replica 4 Way Run IOR and collect BW Run IOR small size and collect IOPS | 1 | 8, 32, 128 | 16, 96, 740 | ||||||||
Any Erasure Encoding object class need to run? EC_2P1G1 | 1? | ||||||||||
| Metadata Test (Using MDTest) | 1 | 1, 8, 32, 128 128 | 1, 16, 96, 256 740 | How many tasks per client 1 ,4 or only 8? What class type should be tested ? -n = 1000 (every process will creat/stat/read/remove ) -z = 0 and 20 (depth of hierarchical directory structure) | Result with 1 server, 1 client is available from https://jira.hpdd.intel.com/secure/attachment/31383/sbatch_run.txt | ||||||
| CART self_test | 2? unless some networking issue we encounter during IOR or functional test | ||||||||||
| POSIX (Fuse) | 2? | 32 | 96 | Run IOR with POSIX mode. Are we there to get the full performance ? | |||||||
| DFS | 2 | Not sure if we want to cover dfs as we are covering daos with IOR on above test cases | |||||||||
| HDF5? | 2? | 32 | 96 | Any specific test we want to run? | |||||||
| FIO? | Do we want to test this? | ||||||||||
| Functionality and Scale testing | Run all daos_test | 128 | 740 | ||||||||
| Single server/Max clients | 1 | 867 | Create pool, Query pool Run IOR (Specific size?) | Poole create should work fine. IOR will run with ~5000 tasks so it should success. Query pool info after IOR run and measure the pool size compare to file size. | |||||||
| Max servers/single client | 867 | 1 | Create pool, Query pool Run IOR | Poole create should work fine. IOR will be run with 8 tasks so it should success. Query pool info after IOR run and measure the pool size compare to file size. | |||||||
Large number of Pools (~1000) | 128 Count is ok? | 740 | Create large number of pools (~90MB each), Write small data with IOR. Restart all the servers. Query all the pools Read the IOR data from each pool with verification what other operation needed after pool creation? | Measure server restart time with this many pools Pool query should report correct sizes after IOR write IOR read should work fine with data validation after all server restart | |||||||
dmg utility testing for example: pool query | dmg pool create dmg pool query dmg pool destroy Anything more to cover? Some of this tools are going to cover in other test cases | ||||||||||
| Negative Scenarios with Scalability | Server failure and rebuild data | 128 | 740 | 1 | Create the multiple pools. Store the IOR with 2,3,4 replica and with multiple groups. Kill server one by one 64 maximum (Half the requested size)? After each server kill read the IOR data and verify the content. Multiple server can be killed (2/4/8), Object data will be lost if all copy lost. May be we can verify the remaining system is functional | Rebuild should happen for all the object and data should not be corrupted after server failure | |||||
| daos_run_io_conf | 128 | 740 | 2 | This will exclude the ranks and add it back in to the loop for given number. We can have maximum 16 targets and include all rank. Test will exclude the rank randomly and add it back. Pool query is also part of this test to verify the usage | We have not tried on TACC but locally it works but there are few issue need to be resolved which we caught during local testing (DAOS-3510) | ||||||
| Reliability and Data Integrity (Soak testing) | |||||||||||
Question:
- Do we need to test other replica other than what mention in above?
- Any Erasure encoding class needs to be tested ?