Middleware Consistency Landing Criteria
Feature Branch Exit Criteria
The following criteria must be met before considering landing the feature branch into master.
Exit Criteria | Status/Notes |
|---|---|
Test automation and CI is 100% complete with “required for version” | Some tests exist, mostly system tests, but tracking % complete pending having a test plan |
No defects with high/critical exposure or “required for version” | None, but may be more filed as testing done |
No new daily or weekly regression test failures (failures not already on master branch) | Analyze latest daily results to ensure any failures are known issues on master (AR: @Phil Henderson ) Perform weekly regression test on feature branch (AR: @Phil Henderson ) |
All tests are passing for feature in weekly and daily regression tests | Perform weekly regression test on feature branch if any daily/weekly tests are added All tests written so far passing and run on PR, not daily or weekly, but pending any additional testing identified in the test plan |
Performance and scale testing executed according to test plan/required requirements. Results are reviewed and approved | Need to perform performance testing Some testing done as reported in final report, but pending any additional testing identified in the test plan |
SDL Code scans are competed (Coverity, Bandit, Checkmarx, Snyk). For all scans, all issues have been dispositioned, and the issues that are required per SDL rules have been fixed. | Run coverity on feature branch (AR: @Ray Dennis (Deactivated) ) Run Snyk on feature branch if not already being done on feature branches by default now (AR: @Ray Dennis (Deactivated) )' Fix all issues needed to be compliant found in said scans |
SAFE review executed if required | SAFE review is not needed |
Any new dependencies identified and Oked by ESAD OSPDT owner | No new dependencies added. |
Does not regress endurance testing from current master baseline with feature enabled | Perform SOAK testing Is this really needed given the code is isolated and not on currently tested code paths? |
Does not regress scale testing from current master baseline with feature enabled | Create a ticket with relevant information (commit/branch to test and any special considerations) and assign to Dalton. Is this really needed given the code is isolated and not on currently tested code paths? |
Manual test plans and “out-of-box experience” testing are 100% executed with “required for version”. Manual test results are recorded | Manual testing done by developer, but pending test plan for additional testing |
Feature branch is up to date with master |
Feature Completion Criteria
The following criteria must be met before considering the feature is complete
Exit Criteria | Status/Notes |
|---|---|
Requirements are defined in SRS Jira Project | Since this was NRE and not in an immediate release, requirements were not input into SRS, but requirements were put into the design document. Does not seem useful use of time to now port to SRS? |
Design document is complete, reviewed, and approved utilizing the team template | Complete: https://daosio.atlassian.net/wiki/spaces/DC/pages/11218419784 |
Test plan for feature is complete, reviewed, and approved utilizing the team template. Security validate plan is also updated if applicable. | Test plan not started and blocked waiting on test resources. (AR: @Ding-hwa Ho ) |
Planned software feature is 100% implemented with “required for version” | Complete. Epic: https://daosio.atlassian.net/browse/DAOS-11599 |
All user documentation updated and reviewed (e.g., admin guide, user guide) | Needs copied from the final report to user guide - but may not want to do this until the feature is officially supported? |
Code coverage results have been reviewed to identify gaps in testing, and test plan has been updated and executed accordingly. Functional and branch coverage each are equivalent or better than master | Not yet measured (AR: @Ding-hwa Ho ) |
Usability is evaluated and results are reviewed | Not started (AR: @Sylvia Oi Yee Chan ) |