This week I read an article called When to Stop Testing (Exit Criteria in Software Testing). The article talks about when to stop testing a software product.
No matter how much we test we can never be absolutely sure that the product is bug free. There is no methods that we can use to prove that a software product is absolutely bug free. The only thing that we can do is iteratively test for all possible defects. However, testing costs money. So when do we stop? Do we stop when we run out of money? Run out of time? Have reach a certain number of bug count? Have fixed a certain number of bugs?
The following are the factors to consider when deciding to stop testing.
Testing can be stopped when:
Requirements:
- 100% Requirements coverage is achieved.
Defects:
- Defined / Desired Defect count is reached.
- All Show Stopper defects or Blockers are fixed and No known Critical / Severity 1 defect is in Open Status.
- All High Priority defects are identified and fixed.
- Defect Rate falls below defined acceptable rate.
- Very few Medium Priority defects are open and have a workaround in place.
- Very few low priority open defects that do not impact software usage.
- All High Priority defects are re-tested and closed and corresponding Regression scenarios are successfully executed.
Test Coverage:
- Test Coverage should be 95% achieved.
- Test case Pass Rate should be 95%. This can be calculated by formula
- ( Total No of TCs Passed / Total number of TCs ) * 100.
- All critical Test cases are passed.
- 5% Test cases can be failed but the Failed Test cases are of low priority.
- Complete Functional Coverage is achieved.
- All major functional / business flows are executed successfully with various inputs and are working fine.
Deadlines:
- Project Deadline or Test Finish deadline is reached.
Test Documents:
- All Test Documents / deliverables (Example – Test Summary Report) are prepared, reviewed and published across.
Budget:
- Complete Testing Budget is exhausted.
From the blog Software Testing – The blog about software by Sudarshan and used with permission of the author. All other rights reserved by the author.