Updated: CI_Weekly_Master_failures.xlsx
Various test runs (e.g. the weekly test run, or master test runs after a merge) are not directly associated with user activity and therefore and no clear responsible party when something goes wrong. With no
...
- Review test run output on a daily basis, any failed runs require follow up.
- Investigate test failures. If a test failure has an existing ticket make sure it is properly identified as an "Intermittent Test Issue". If there is not an existing ticket, create one. Use the template below when creating the ticket.
- Review CI test issue tickets using the pre-defined Jira query "Intermittent Test Issues".
- Bring failures that do not receive prompt action to the attention of management and/or the daos triage team.
- Tickets that have been open for 30 days or more and have "Number of Occurrences" set to 1 are candidates for closure. Follow up with assignee and/or triage group team to reevaluate.
- Present a short, reoccurring segment at the Monday test meeting on CI triage status.
Non-PR Testing Rotating Owners
Rotation begins WW30'20. The rotation term is 2 weeks. Owners are responsible for finding or exchanging with an alternate if their rotation falls during a vacation or other OOO situation.
| Owner |
|---|
| Sylvia |
| Saurabh |
| Ding |
For latest update on ticket status, full list of intermittent failures and details on failures, please refer to ticket in JIRA using filter "Intermittent Test Issues"
Use the template below for creating a new ticket for tests failing in CI.
...
Labels: Intermittent Test Issues (for PR/master failures) or weekly_failures (for failures on weekly-testing branch)
Fill in Bug Exposure, Bug Quality, Bug Type accordingly.
Summary: Provide a summary of the issue (if it is an issue on weekly, precede with "Weekly Test - <summary>"
Description: Please include the following in the description.
...
Include stack trace, error message
Attach debug logs.
Intermittent CI failures
Here is a quick summary of recent CI failures.
For latest update on ticket status, full list of intermittent failures and details on failures, please refer to ticket in JIRA using filter "Intermittent Test Issues"
...
| Jira Legacy | ||||||
|---|---|---|---|---|---|---|
|
...
Test died without reporting the status. Runner error occurred: Timeout reached
Test should have produced a coredump.
...
| Jira Legacy | ||||||
|---|---|---|---|---|---|---|
|
...
Test died without reporting the status. Runner error occurred: Timeout reached
Test should have produced a coredump
...
| Jira Legacy | ||||||
|---|---|---|---|---|---|---|
|
...
| Jira Legacy | ||||||
|---|---|---|---|---|---|---|
|
...
.
...