Running DAOS Functional Tests

Owned by Scott Kirvan (Unlicensed)
Last updated: Sept 21, 2021 by Dalton BohningVersion comment
  • Tests are run using a test framework called Avocado.  The version known to work with the existing tests is 69.3; if you yum install on Boro that should be the version you get.
  • To install and run tests from RPMs, see Manually Running Functional Tests from RPMs.
  • To install and run tests from a local build, see Use the local workspace for running functional tests.
  • Tests are in $PREFIX/lib/daos/TESTING/ftest.
    • In an RPM env, $PREFIX=/usr
    • In local build env, $PREFIX=myrepo/install.
  • Sub-directories contain categories of tests. E.g. the pool directory contains tests related to pools.
  • A given group of tests is implemented as a combination of a python file (.py) and a yaml file (.yaml). For example, in the pool directory there is a SimpleCreateDeleteTest.py and SimplecreateDeleteTest.yaml.  The python contains the test code that drives the test, and the yaml file contains the test parameters.  A single function in the python file can execute a large number of test cases because it's run for different combinations of inputs as found in the yaml file.
  • For tests that run on multiple servers, these two files need to be accessible by remote server nodes:
    • daos_avocado_test.yaml
    • daos_server.attach_info_tmp

           In a local build env, they are located in myrepo/install/tmp.

           In RPM env, they are located at DAOS_TEST_SHARED_DIR if this env variable is set; otherwise,  by default thy will be installed at ~/daos_test This directory will be automatically created if it does not exist..

  • Before running a test, you must identify the number of cluster nodes that are required.  To do this, open the yaml file for the test you want to run ($PREFIX/lib/daos/TESTING/ftest/*/*.yaml).  You’ll see placeholders for machine names. E.g. server-A.  If the test requires more than 1 host you’ll see server-A and server-B, etc. You'll need to have enough servers/clients depending on how many are specified in the yaml.

  • Tests are started with the launch.py script in the $PREFIX/lib/daos/TESTING/ftest directory along with a test tag.  Groups of tests are identified by tags.  For example, tests in $PREFIX/lib/daos/TESTING/ftest/pool/simple_create_delete_test.py are tagged with all, pool, and simple_create. The all tag is associated with all tests, the pool tag is associated with most pool-related tests, and the simple_create tag is associated with only the tests in this file. To run the simple_create tests, execute:

    ./launch.py -ts boro-xx -tc boro-xx simple_create

    See Using launch.py for more details on running tests.

    If you were to make the tragic mistake of specifying the all tag, the tests would run for a day at least.



TODO:

  • Add section and instructions for generating a public/private key pair and using ssh-copy-id to install it on all machines you will be using in the test suite.
  • Add instructions on how to add passwordless sudo on the accounts the launcher will be run on for the target hosts.
  • Add instructions based on Distro of which files to copy from utils/systemd into /usr/lib/systemd to allow the test harness to use systemctl and how to modify them to run as your user.
  • Move these instructions into a file doc/dev/testing.md to make it accessible to other developers.
  • Add instructions on creating /etc/daos paths needed since test harness does not use sudo on mkdir commands even though it uses sudo on cp commands.
  • Add instructions to create all of the necessary groups needed by the unit files