Test Tags

What Are Test Tags?

Functional tests use Avocado, which uses tags to label tests. A test can have many tags, and a tag can belong to many tests. These tags are specified in a docstring comment for each test function. For example:

def test_sample_two(self):
  """Another sample test.
  :avocado: tags=all,full_regression
  :avocado: tags=hw,large
  :avocado: tags=feature1,feature2
  :avocado: tags=sample,sample_one
  """

How Do I use Test Tags in a PR?

The /wiki/spaces/CI/pages/1389759093 are used to specify which tests are ran for a PR. By default, all tests tagged with pr are ran. I.e. Test-tag: pr. To run specific tests, you should first identify which tags you need. See Feature Tags for a list of common tags. There are two ways to specify which tags to run:

Test-tag - Runs only the tags specified.
1. Test-tag: pr checksum - Runs pr tests and checksum tests.
2. Test-tag: checksum - Runs only checksum tests.
Features - Runs the default set of tags (pr), plus the tags specified minus full_regression.
1. Features: checksum - Equivalent to Test-tag: pr checksum,-full_regression

Whether to use Test-tag or Features mostly depends on whether you want to include the full_regression tests for the specified feature tag. In the future, full_regression tests will not be excluded. Regardless, pr tests are almost always required to be successfully ran for a PR to be merged.

List of Frequency Tags

These tags determine when a test runs. Some tests might be ran at multiple frequencies.

Avocado Tag	Description
`pr`	Ran for every PR. These are generally small core tests.
`daily_regression`	Ran once daily. These are generally less critical than `pr` tests.
`full_regression`	Ran weekly. These are generally less critical than `daily_regression`, or longer-running tests. These usually do not overlap with `daily_regression` tests.
`manual`	Manual tests for special purposes. These tests might: not run in CI, require special configuration, be for specific environments.
`soak`	SOAK tests are ran manually at specified intervals.

List of Stage/Environment Tags

These tags determine where a test runs. Each test should only have one set of these tags.

Avocado Tag	Description
`hw,small`	Tests that are ran on the Small Hardware Functional Test stage
`hw,medium,ib2`	Tests that are ran on the Medium Hardware Functional Test stage.
`hw,large`	Tests that are ran on the Large Hardware Functional Test stage.
`vm`	Tests that are ran on the VM Functional Test stages. This tag is usually not specified explicitly. Instead, the lack of a `hw` tag represents VM tests.

List of Feature Tags

These tags group tests by common Features. This list is currently incomplete.

Avocado Tag	Description
`checksum`	Tests that verify or use `checksum` features.
`control`	Tests that verify `control`plane features.
`datamover`	Tests related to Data Mover utilities, including `dcp`, `dsync`, `daos fs copy`, `daos cont clone`, `daos-serialize`
`daosperf`	Tests that verify `daos_perf`.
`dfuse`	Tests that verify or use `dfuse`.
`ec`	Tests that verify `ec` features.
`ior`	Tests that verify or use `ior`.
`mdtest`	Tests that verify or use `mdtest`.
`osa`	Tests that verify online and offline `osa` features.
`security`	Tests that verify `security` features.
`snap`	Tests that verify `snapshot` features.
`tx`	Tests that verify `transaction` features.