A topic we touched upon in class for a tiny bit was mutation testing. It is a different kind of software testing, where it alters your code and then runs your tests. If your tests fail, that means your tests are good. If your tests pass, that means your tests are not good. The altered versions of the code are called mutants, and if they are caught by the tests, as in failed, they are considered killed. If they pass, they are considered to have survived. The concept is strange, but can be effective, especially in finding inaccurate tests or seeing if you have enough tests.
In this blog post, Uncle Bob talks about his experience with mutation testing, how he came upon it, some benefits and how it works. He starts off by stating some issues with unit testing, such as writing enough tests, covering every line, branch, or path, and whether or not a test will fail depending on if you change some code. He answers these issues with sufficiency, coverage, and semantic stability, all of which can be solved by using mutation testing. It is fairly simple to run, and there is multiple ways to go about it. You run pitest tool, which alters your code a little bit, such as inverting if statements. It will do this more than once, where each variation of your code is a mutant. Then, for each mutant, it will run through all of your tests. If the tests fail, the mutant is killed, and if the tests pass, the mutant survived. Ideally, you want all of the tests to fail, which may be difficult to wrap your head around, typically you want your tests to pass. Having mutation testing is just another layer of testing to ensure your code is working properly. If a mutant survived, a number of things could have happened, such as ignored tests or discipline became too relaxed and people started writing sloppy code. Either way, it’s a good way to reinforce your tests and code.
We did not really get to play around with this a lot in class, but it was cool to see how it worked. It had created multiple batches of mutants and might have ran the tests individually, but I may be wrong about that. If we had the chance to run it on code that we wrote, or if I had ran it on code that I wrote, maybe I could have understood it more. I will definitely use this in the future.
From the blog CS@Worcester – Cao's Thoughts by antcao and used with permission of the author. All other rights reserved by the author.