Versions Compared

Version Old Version 22 New Version 23
Changes made by

Dalton Bohning

Dalton Bohning

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

The /wiki/spaces/CI/pages/1389759093 specify which tests are run for a PR. By default, all tests tagged with pr are ran. I.e. Test-tag: pr. To run specific tests, you should first identify which tags you need. See Feature Tags for a list of common tags. To run a specific test case, you will need to find its tag in the test file. There are two pragmas to specify which tags to run:

  1. Test-tag - Runs only the tags specified.

    1. Test-tag: pr checksum - Runs pr tests and checksum tests.

    2. Test-tag: checksum - Runs only checksum tests.

  2. Features - Runs the default set of tags (pr), plus the tags specified minus full_regression.

    1. Features: checksum - Equivalent to Test-tag: pr checksum,-full_regression

...

  1. Test-tag

...

  1. - Runs only the tags specified.

    1. Test-tag: pr checksum - Runs pr tests and checksum tests.

    2. Test-tag: checksum - Runs only checksum tests.

Info

pr tests are almost always required to pass for a PR to be merged. The Features tag should be used in most cases.

When Do I Need to Specify Test Tags?

  1. When a change is 100% covered by existing unit tests, there is no need to specify additional tags.When a change is covered by new unit tests, you might need to specify the corresponding Feature Tag.

  2. When a change is not 100% covered by unit tests, you do need to specify the corresponding Feature Tag.

    1. E.g. Middleware applications, high-level functional verification, 3rd-party libraries, integration

  3. When a change modifies or adds Functional Tests or the Functional Test Framework, you do need to specify the corresponding Feature Tags and/or Specific Test Tags.

  4. When a change fixes a failing Functional Test, you do need to specify the corresponding Specific Test Tags.

...