In this article, James Bach talks about the ways in which conversations about testing skew away from the reality of what it means to actually do software testing, and what it means to be a skilled tester. As is clear from the title, he divides this broad topic into six smaller pieces:
- Size doesn’t matter. The number of test cases that you run is not a meaningful number, just like the number of lines of code in a project is not meaningful. What matters is what they cover and what they can teach us.
- Tests are performances. It’s an activity, not an object. The person implementing test cases is more important than the cases themselves.
- Testing strategy needs to evolve. Testing is a process of constant interrogation (I’ve written about this before in response to other articles) of both the code and the tester.
- Automation does not define testing. Automation is a tool with which to run tests. It takes human judgment and skill to design and implement quality tests. Automation is a way to do a lot, but it takes a tester’s skills to do a lot with as little as possible.
- There are many kinds of test coverage. No one type of coverage is truly comprehensive, and making changes to tests to give additional coverage by one metric may change what is covered by another metric.
- Testing is not static. It’s an activity that’s fundamentally about learning. Some things are predictable, but many things are not. Testing is essetially formulating and then running experiments. Much can be gained by deviation from established procedures.
The biggest takeaway for me from this article is the notion of testing as design and execution of experiments. It’s not something I’d ever really thought about before, and it makes a lot of sense. Testing is the process of formulating a hypothesis (these inputs in this context should result in this output) and then trying it out to see whether it’s right. It even involves fairly thorough hypothesis testing, so that we can claim with confidence that a particular outcome of use is guaranteed (or close enough).
This article is also a discussion of the ways in which one can have a useful conversation about testing, and the ways in which those conversations can turn useless. I think this is important to take away too, when I’m looking for jobs in the software development field. I want to avoid making conversational mistakes in interviews by going down the wrong path, and I also may want to avoid working at a company where those conversational mistakes are part of the corporate culture.
From the blog CS@Worcester – orscsblog by orscsblog and used with permission of the author. All other rights reserved by the author.