Internal bug finding, where project developers find bugs themselves, can be superior to external bug finding because developers are more in tune with their own project’s needs and also, they can schedule bug finding efforts in such a way that they are most effective, for example after landing a major new feature. Thus, an alternative to external bug finding campaigns is to create good bug finding tools and make them available to developers, preferably as open-source software. Bug finders are motivated altruistically (they want to make the targeted software more reliable by getting lots of bugs fixed) but also selfishly (a bug finding tool or technique is demonstrably powerful if it can find a lot of previously unknown defects). A priority for “a valid bug that should be fixed, but” I’ve found FindBugs’ “Mostly Harmless” useful for this and have lobbied to include it in every bug tracker I’ve used since, though it’s more of a severity than a priority. External bug finders would like to find and report as many bugs as possible, and if they have the right technology, it can end up being cheap to find hundreds or thousands of bugs in a large, soft target. A major advantage of external bug finding is that since the people performing the testing are presumably submitting a lot of bug reports, they can do a really good job at it.
A priority for “a valid bug that should be fixed, but not something that will ever be at the top of the list unless something unforeseen happens.” The OSS-Fuzz project is a good example of a successful external bug finding effort; its web page mentions that it “has found over 20,000 bugs in 300 open-source projects. As an external bug finder, it can be hard to tell which kind of bug you have discovered (and the numbers are not on your side: the large majority of bugs are not that important). However, if they actively don’t trust you, for example because you’ve flooded their bug tracker with corner-case issues, then they’re not likely to ever listen to you again, and you probably need to move on to do bug finding somewhere else. Much more recently, it has become common to treat “external bug finding,” looking for defects in other people’s software, as an activity worth pursuing on its own. An all-too-common attitude (especially among victims of bug metrics) is that bug reporters are the enemy—but those making pull requests are contributors. First, every bug report requires significant attention and effort, usually far more effort than was required to simply find the bug. Occasionally, your bug finding technique will come up with a trigger for a known bug that is considerably smaller or simpler than what is currently in the issue tracker.
From the blog CS@Worcester – The Dive by gonzalezwsu22 and used with permission of the author. All other rights reserved by the author.