Test Tags

What Are Test Tags?

Functional tests use Avocado, which uses tags to label tests. A test can have many tags, and a tag can belong to many tests. These tags are specified in a docstring comment for each test function. For example:

def test_sample_two(self):
  """Another sample test.
  :avocado: tags=all,full_regression
  :avocado: tags=hw,large
  :avocado: tags=feature1,feature2
  :avocado: tags=sample,sample_one
  """

How Do I use Test Tags in a PR?

The /wiki/spaces/CI/pages/1389759093 are used to specify which tests are ran for a PR. By default, all tests tagged with pr are ran. I.e. Test-tag: pr. To run specific tests, you should first identify which tags you need. See Feature Tags for a list of common tags. To run a specific test case, you will need to find it’s tag in the test file. There are two pragmas to specify which tags to run:

Test-tag - Runs only the tags specified.
1. Test-tag: pr checksum - Runs pr tests and checksum tests.
2. Test-tag: checksum - Runs only checksum tests.
Features - Runs the default set of tags (pr), plus the tags specified minus full_regression.
1. Features: checksum - Equivalent to Test-tag: pr checksum,-full_regression

Whether to use Test-tag or Features mostly depends on whether you want to include the full_regression tests for the specified feature tag. In the future, full_regression tests will not be excluded. Regardless, pr tests are almost always required to be successfully ran for a PR to be merged.

List of Frequency Tags

These tags determine when a test runs. Some tests might be ran at multiple frequencies.

Avocado Tag	Description
`pr`	Ran for every PR. These are generally small core tests.
`daily_regression`	Ran once daily. These are generally less critical than `pr` tests.
`full_regression`	Ran weekly. These are generally less critical than `daily_regression`, or longer-running tests. These usually do not overlap with `daily_regression` tests.
`manual`	Manual tests for special purposes. These tests might: not run in CI, require special configuration, be for specific environments.
`soak`	SOAK tests are ran manually at specified intervals.
`deployment`	Ran when deploying a new DAOS installation.

List of Stage/Environment Tags

These tags determine where a test runs. Each test should only have one set of these tags.

Avocado Tag	Description
`hw,small`	Tests that are ran on the Small Hardware Functional Test stage
`hw,medium,ib2`	Tests that are ran on the Medium Hardware Functional Test stage.
`hw,large`	Tests that are ran on the Large Hardware Functional Test stage.
`vm`	Tests that are ran on the VM Functional Test stages. This tag is usually not specified explicitly. Instead, the lack of a `hw` tag represents VM tests.

List of Feature Tags

These tags group tests by common Features. This list is currently incomplete.

Avocado Tag	Description
`checksum`	Verifies `checksum` features.
`container`	Verifies `container` features.
`control`	Verifies `control`plane features.
`datamover`	Verifies Data Mover utilities. Additional sub-tags: `fs_copy` - `daos filesystem copy` tests `cont_clone` - `daos container clone` tests `dcp` - `mpifileutils/dcp` (distributed copy) tests `dsync` - `mpifileutils/dsync` (distributed sync) tests `dserialize` - `mpifileutils/daos-serialize` and `mpifileutils/daos-deserialize` (distributed serialization) tests
`daosperf`	Verifies `daos_perf`.
`deployment`	Verifies DAOS on a newly deployed system.
`dfuse`	Verifies `dfuse`.
`ec`	Verifies `ec` features.
`ior`	Verifies `ior`.
`mdtest`	Verifies `mdtest`.
`osa`	Verifies online and offline `osa` features.
`security`	Verifies `security` features such as ACLs, pool and container permissions.
`snap`	Verifies `snapshot` features.
`telemetry`	Verifies `telemetry` features.
`tx`	Verifies `transaction` features.