Updated: CI_Weekly_Master_failures.xlsx
Various test runs (e.g. the weekly test run, or master test runs after a merge) are not directly associated with user activity and therefore and no clear responsible party when something goes wrong. With no
...
- Review test run output on a daily basis, any failed runs require follow up.
- Investigate test failures. If a test failure has an existing ticket make sure it is properly identified as a an "CI Intermittent Test Issue". If there is not an existing ticket, create one. Use the template below when creating the ticket.
- Review CI test issue tickets using the pre-defined Jira query "Intermittent Test Issues".
- Bring failures that do not receive prompt action to the attention of management and/or the daos triage team.
- Tickets that have been open for 30 days or more and have "Number of Occurrences" set to 1 are candidates for closure. Follow up with assignee and/or triage group team to reevaluate.
- Present a short, reoccurring segment at the Monday test meeting on CI triage status.
Non-PR Testing Rotating Owners
Rotation begins WW30'20. The rotation term is 2 weeks. Owners are responsible for finding or exchanging with an alternate if their rotation falls during a vacation or other OOO situation.
| Owner |
|---|
| Sylvia |
| Saurabh |
| Ding |
For latest update on ticket status, full list of intermittent failures and details on failures, please refer to ticket in JIRA using filter "Intermittent Test Issues"
Use the template below for creating a new ticket for tests failing in CI.
Project: <CaRT | "CORAL - CI" | DAOS>
Issue type: Bug
Labels: Intermittent Test Issues (for PR/master failures) or weekly_failures (for failures on weekly-testing branch)
Fill in Bug Exposure, Bug Quality, Bug Type accordingly.
Summary: Provide a summary of the issue (if it is an issue on weekly, precede with "Weekly Test - <summary>"
Description: Please include the following in the description.
Failed Stage: < Build | Unit Test | Test>
Failed Build/Test:
For build failure, please list the build stage that failed e.g. "Build RPM on CentOS 7", "Build RPM on Leap 15", etc
For test failure, please list the test folder, source (and variant when possible) e.g. daos_test/daos_core_test.py - DAOS degraded-mode tests
Branch: <master | name_of_branch>
Commit: <commit hash>
Include stack trace, error message
Attach debug logs.