Category Archives: CS-443

More on Static Testing: Bug-Finding Tools

In my previous blog for CS-443, I discussed my experience revisiting the class activity about static testing tools. I focused on figuring out the Checkstyle tool, which makes sure that the code complies with a set of style guidelines specified in an xml file. However, the class activity dealt with more tools than just Checkstyle. Today, I would like to look at the second part of this activity, which deals with tools that detect actual bugs rather than simple style issues.

The bug-finding tools that the activity focuses on are FindBugs, SpotBugs, and PMD. Much like Checkstyle, these three tools are extremely easy to add to a Gradle project by adding a few new lines to the build.gradle file. They each require just a single line to apply them as a plugin for the project. In addition, a small block of text can be used to set different properties for each tool, such as their version number. The activity also recommended adding lines to the options for FindBugs and SpotBugs to report their findings in an html file, since they use an xml file by default. This made the errors much more readable and easier to understand. Finally, a single line must be added to the Dependencies for SpotBugs to function.

Once build.gradle is properly configured, the tools are run simply by building the project (or calling “gradle check,” as I discovered). All three tools will then analyze the code and create reports explaining the types of errors they find. I ran the tools on the code provided for the activity, and I was surprised by how useful their reports were. They point out code that follows bad practice, code that may cause compatibility issues on different platforms, and even code that may negatively impact performance. I find it interesting that these tools are able to detect such errors without running the code, and I definitely see them being extremely useful as I often do not detect such errors myself even after hours of searching for problems manually.

Since these tools are so effective at finding errors, I was curious if there was even any benefit to using manual code review over one of these tools. I did a bit of research into this, and I found a blog post on synopsis.com that I think makes a great point – that these tools are unable to understand the context of potential errors in the same way that a human can. The blog also lists eight major limitations of these tools that should be considered when using them over manual review. Although static testing tools are able to find error in code quickly and easily, it is still necessary for the developers to determine whether the errors detected are valid or useful.

Link to the blog:

https://www.synopsys.com/blogs/software-security/static-analysis-tools-finding-bugs/

From the blog CS@Worcester – Computer Science with Kyle Q by kylequad and used with permission of the author. All other rights reserved by the author.

Regression Testing

Article link: https://dzone.com/articles/what-is-regression-testing-and-why-is-it-important

Today I learned about regression testing from an excellent article on DZone and why it is an essential part of software testing. As the article explains, regression testing involves testing the whole software product to ensure that any changes to the code doesn’t break what was already working in the product or other parts of the software. The article then goes through a great example of how fixing a bug in one part of an example piece of software can unintentionally result in another (previously working) part of the system to stop working properly. The article demonstrates really well that this breakage can occur despite unit tests showing that the bug was fixed properly. This to me is an especially important point as it shows the value in using multiple methods of testing to ensure software is performing correctly. One criticism I do have of this article is that I wish they gave an example of how to implement regression (the article does include a link to a tool that performs regression testing) testing in an actual program (or in context of their previous example).

Although after reading this article, the concept of regression testing seems simple, I find it very important as both a software user and software developer. In my personal experience as a user I have seen everything from operating systems to video games release updates that caused features that worked fine previously to become buggy or stop working entirely. From the user perspective I know how frustrating this can be, so I am glad that I learned how to prevent this problem as a software developer. Regression testing is definitely something I now want to implement into my personal software projects. Now I need to look into the tools that can help do this.

From the blog CS@Worcester – Chris' Computer Science Blog by cradkowski and used with permission of the author. All other rights reserved by the author.

Mutation testing

Time to read up on mutation testing. Jasper Sprengers’ article, “Sensible mutation testing: don’t go on a killing spree,” gives a small overview of mutation testing and then explains how killing all mutants is not always necessary. The article uses an example to show how a simple class requires extensive testing to kill all mutants. The example shows how a small class of around 10 lines and a single method requires 18 test assertions to kill all mutants. This shows that it is important to consider letting some mutants live. It is not always efficient to use the extra time expanding the tests. My takeaway from this article is that mutation testing should be used to find situations where your tests are not thorough enough but not all mutants are worth the time killing. Use mutation testing with discretion.

Article referenced:
https://blog.codecentric.de/en/2016/02/sensible-mutation-testing-dont-go-killing-spree-2/

From the blog CS@Worcester – D’s Comp Sci Blog by dlivengood and used with permission of the author. All other rights reserved by the author.

Test Doubles

Test Doubles is a term used to describe code tests that are basically imposters of themselves. They are unfinished and simplified versions of what the actual tests would be. Thus, they are doubles of their true forms. Theses doubles are written in order to satisfy and verify that you code is functioning and is setup at the most basic level. Test Doubles contain three different types; Stubs, Mocks, and Fakes. Each of these types is used in a different way to help get your testing going but without getting too complicated. 

A fake test allows us to take a shortcut when testing functionality. Say you have a database that your code connects to. Instead of starting up and running a connection you could have hard coded values in your code that you retrieve for your tests. So in this way you aren’t really testing the right values but that your code is able to get values at all.

A stub has a similar idea with using hard coded values to satisfy your tests. Essentially all you do is create a test, then make sure that the code that is being tested has exactly the results that are expected. For example, if you’re testing if a rectangle object is a square then you don’t actually create the rectangle, instead you return the expected value to the test class.

A mock is used when we just want to verify that our code is being called or accessed correctly without actually functioning the way we are intending. If you have a method you want to make sure is called correctly you can have that method be empty but when it is called it prints a string “Method … accessed correctly”. This way you know that you are calling the right method without having to actually test its functionality yet.

Here is a link that helps explain these Test Doubles:

https://blog.pragmatists.com/test-doubles-fakes-mocks-and-stubs-1a7491dfa3da

From the blog cs@worcester – Zac's Blog by zloureiro and used with permission of the author. All other rights reserved by the author.

Path Testing

Path testing is a great way to ensure that your code is concise and understandable. It allows you to see the flow of your code and the direction in which it travels depending on conditions, loops, and the order of the code. The name is self explanatory you are essentially testing the “path” of the code. This kind of testing is not in the same field as something like a Junit test. This is a visual test that doesn’t use software persay. It’s more like a diagram that allows you to test and figure out the logic of your code. Here is a diagram I found online that helps explain this type of testing.

In this diagram you can see that the numbered nodes match up with the lines of code. This helps to make the details of the code abstract and to bring the path of the code into the forefront. This isn’t very detailed code but it shows how conditionals effect the path of your code. Certain nodes are passed through and others are not depending on these conditionals. Setting up these diagrams for your code can help you understand your code better and thus eliminate redundant testing or create more useful tests.

Here’s a link to a website giving a good explanation of path testing.

https://www.guru99.com/basis-path-testing.html

From the blog cs@worcester – Zac's Blog by zloureiro and used with permission of the author. All other rights reserved by the author.

The Last Blog, Ultra Efficient Computers

For what may possibly be my last ever blog post on this blog, I found an article that discusses the possibility of Ultra-efficient computers using Atomic scale manufacturing. That sentence alone is enough to grab anyone’s attention (It certainly got mine.) After reading the introduction, I discovered that this article is about saving the environment rather than just having really fast computers. However, that’s still great because something needs to be done about the environment and this could be it. The article states that today’s computers require enough power to release more than 1 gigatonne of carbon emissions per year. That is actually really bad.  ACS Nano has a solution though. They are making computers that store more data, and use less power. You would think that this wouldn’t be possible without some kind of trade off, but they figured it out.

The attained this by manipulating singular atoms in order to produce “ultra dense memory arrays” which can store way more data in a smaller space. They have ran into an issue where bottleneck is apparent, so they are still trying to find a way to make this process more efficient. In order to conduct this process, scientists must use a technique called hydrogen lithography. This is a process in which they remove certain hydrogen atoms from a silicon surface in order to write more data. They demonstrated this technique on a 24-bit memory array, and the result was a 1000 times faster fabrication of atomic computers. This means that “real world” manufacturing can begin. According to ACS, this method would consume 100 times less power, making it a huge step in the right direction towards a cleaner Earth.

It was a pleasure reading this article considering it was very short and it had a lot of interesting information on it. I didn’t expect so many chemistry topics to be involved, but I love chemistry so that is okay. This will probably be my last blog post ever, so to my readers, you have been a great audience. Thank you.

https://www.sciencedaily.com/releases/2019/11/191127090225.htm

From the blog CS@Worcester – My Life in Comp Sci by Tyler Rego and used with permission of the author. All other rights reserved by the author.

Path testing

Path testing is widely used to design test cases. Path testing process has 4 steps, which is to draw control flow graph, calculate Cyclomatic Complexity, make set of paths, and then create test cases for the those paths, which would use this formula: E – N + 2P (where E is number of edges, N is number of vertices and P is program factor). Path testing usually use control flow graph, which would help developers find sets of linearly dependent paths of execution. Path testing also use Cyclomatic Complexity to determine the number of linearly independent paths and each path should be a separate test case.

Besides control flow graph, path testing can also use different techniques like decision to decision path, where control flow path can be broken into various decision to decision paths and collapse into individual nodes, and Independent paths. There are various advantages of path testing like making sure that tests are isolated and not redundant to each other, it helps developers focus more on the logic of the programs, and finally, it helps developers to design test cases in a much easier and simpler way.

Article can be found here.

From the blog #Khoa'sCSBlog by and used with permission of the author. All other rights reserved by the author.

Code coverage

Code coverage is a subject that has recently come up in my Testing class and it did catch my attention for its useful functionality. So what’s code coverage? Code coverage is how much of the code that has been executed during the testing process. This process is not only check for every line of code, it also checks if all the branch of conditional and loops is covered. With this process in place, it would surely decrease the number of bugs.

With that being said, how would developers be able to apply this into their code? There are a lot of system and plugins out there that would help doing this job correctly, I will take JaCoCo plugins for Gradle as an example in this post, since I think it is a really good system that gives excellent reports. To enable JaCoCo, add this line to build.gradle:
 apply plugin: ‘jacoco’ 
What is greater about JaCoCo is that it let user define thresholds or conditions that the code coverage has to pass in order for the build system to return pass. For example, consider this configuration below:
jacocoTestCoverageVerification {
    violationRules {
        rule {
            limit {
                minimum = 0.8
            }
       }
    }
}
check.dependsOn jacocoTestCoverageVerification
When this task runs, build system will ensure that the build only pass only if the code coverage metric reaches 80%. As simple as it looks like, I think it is an essential quality of life plugin to have for the build system.
Article can be found here.

From the blog #Khoa'sCSBlog by and used with permission of the author. All other rights reserved by the author.

Mockito, not the lovely beverage Mojito

Does your unit test have a stunt double? You know someone to jump in there and take the hit during testing instead of using real dependencies? Well, it could if you used Mockito. I found a great tutorial on setting up Mockito for various environments and situations here: https://www.vogella.com/tutorials/Mockito/article.html But what is Mockito? Don’t confuse it with that lovely mint based rum concoction. It’s a JAVA based mocking framework. It’s used to mock interfaces so that dummy functionality can be used in unit testing. But what is mocking? In OOP mock objects are simulated objects that mimic the behavior of real objects (stunt doubles!). In Test Driven Development mock objects meet interface requirements of real objects allowing developers to write and unit-test functionality. This allows the developers to focus on the behavior of the system while testing without worrying about dependencies. This Martin Fowler article does a great job explaining Mocks: https://martinfowler.com/articles/mocksArentStubs.html So grab a Mojito and enjoy some reading! #CS@Worcester #CS-443

From the blog Michael Duquette by Michael Duquette and used with permission of the author. All other rights reserved by the author.

Gradle Clover Part 2

Before we get started there a couple of assumptions I am making: You have a current Gradle project and you want add Code Coverage testing You are using Junit 5 (Jupiter) for testing Ok first and for most backup your existing project. If following my steps causes you to screw something up that’s on you. Especially since my first step in this adventure clearly states backup your existing project! Backup complete? Ok lets move on. Go to github and fork the gradle-clover-plugin repository: https://github.com/bmuschko/gradle-clover-plugin Now open your projects build.gradle and the plugins build.gradle. Wildly different right?! Now merge the two. My project didn’t have any funky dependencies or imports so I literally copied the plugins build.gradle and overwrote mine. There are about nine different plugins used for the Gradle Clover Plugin. Read the projects build.gradle and understand it. This will help assure there are no failures. Remember I did tell you to backup your project before starting. Once you sort out your build.gradle look in the Gradle Clover Plugin repository and open up the gradle folder. Copy the six *.gradle files to your projects gradle folder. These should be: additional-artifacts.gradle documentation.gradle functional-testing.gradle integration-test.gradle publishing.gradle release.gradle Now back to the Gradle Clover Plugin repository copy the src folder to your project and overwrite your src folder. You will now have three folders in your src folder: funcTest main test funcTest contains all the functional testing scripts while main and test contain an additional groovy folder with scripts specific for Groovy. Now if you just overwrote your src directory your code should still be in the java folders under main and test. If not go to your backup and copy them over. This is why we do backups. Now once again back to the Gradle Clover Plugin repository. Copy the gradle.properties file over from the root of the repository to the root of your repository. Open it up with your favorite editor (I like to use Notepad++ ) and check it out. What are we looking at here? Well this sets your current Gradle version build level and the testing versions as well. You can modify this to test against specific versions of Gradle. TIP: Keep gradleTestingVersion list short and only test against the version of Gradle you need to. This will reduce your build time. Now open a bash terminal and fire off a gradle build. Oh your code failed with a compilation error because of javaDocs and GroovyDocs? Clean up your code! No seriously, I was shocked the first time I ran it that I had 12 errors all related to java docs. Now these were errors that JaCoCo, Spotbugs, Checkstyle and PMD did not report. One of which was: @param firstName was missing a description. It was helpful to have the line numbers right there on the screen and I was quickly able to resolve my issues. Now that you’ve cleaned up your code do another gradle build. Once your build succeeds navigate to the $repository_name/build/reports/tests folder you will see two sub-folders test and functionalTest. Drill into these folder and open the index.html file to see your test results. Now that you are all setup head over to OpenClover and to the gradle-clover-plugin repository and learn how to tweak the setup for your use. #CS@Worcester #CS-443

From the blog Home | Michael Duquette by Michael Duquette and used with permission of the author. All other rights reserved by the author.