In episode 31 of a podcast by Brian Okken on his show titled “Test and Code,” Brian has a discussion with guest Paul Merrill about the testing pyramid and why he is frustrated with certain test-driven development (TDD) models. The discussion began as a Twitter disagreement between the two test enthusiasts and blossomed into a full-fledged “civil discussion.” While both Brian and Paul agree about the importance of testing and see the value in test-driven development, they disagree about how extensive testing needs to be in order to be effective. The two also disagree about how tests should be written and to what extent code should be based on testing versus tests written based on code. Although they do not seem to ever reach a consensus on the issue, each of them make some excellent points and give examples of personal experiences to support their opinions.
Brian, for example, is fed up with the sheer number of redundant unit tests that are written to test the same thing in traditional pyramid testing. Brian presents an example of a hypothetical method that has a test written for handling negative numbers, but the higher-level method that passes values to the first method will never pass a negative value. Brian sees testing the first method’s ability to handle negative numbers as unnecessary and a waste of time. If these same types of tests are written for hundreds of methods, Brian argues, an extraordinary amount of time is wasted on useless testing. As a rebuttal, Paul argues that if changes are made to the higher-level method that allows negatives to be passed down, the tests would already be written. Clearly not convinced, Brian scoffs at this justification for the test he clearly sees no value in.
Paul seems to have some sort of personal experience in all of the obscure areas that Brian dismisses as rare or unusual scenarios. Paul seems to support a bottom-up test-driven development platform, where tests are written that outline how every last detail that a program will perform. Paul argues that this is the best way for tests to effectively drive development. He seems to think that tests are not able to aid in the development process if they are written after development has taken place. Brian, on the other hand, sees this issue from a top-down perspective. He argues that higher, user-level tests should be written first. When the high-level tests are insufficient to further drive development or are too ambiguous to write code for, then lower-level unit tests should be written.
It was extremely interesting to listen to two experienced testers discuss such a controversial topic. While I can see where both men are coming from with their opinions, I seem to lean towards Brian’s side in the argument over test-driven development. I think that the points he makes about a pragmatic approach to testing is important. Rather than generating some ridiculous number of unit tests that may not have any bearing on the actual functioning of the program, the effort should be put into doing what was originally promised by the program specification. I think that I will follow Brian’s advice and also take the pragmatic approach to writing tests, in the hopes of avoiding rewriting tests and code. In the end, its not about how many tests can be written, its about testing for the correct things in order to deliver a bug-free program.
From the blog CS@Worcester – ~/GeorgeMatthew/etc by gmatthew and used with permission of the author. All other rights reserved by the author.