Category Archives: CS-443

CS443 – Introduction

Hello everyone, we are back and doing it again this semester. Tech Worth Talking About returns with posts about two courses, CS443 – Software Quality Assurance and Testing as well as CS448 – Software Development Capstone.
CS443 will focus more on strategies and tools for automating code testing as well as how testing incorporates into the overall development process. This goes far beyond simply debugging.

Welcome, and stay tuned!

From the blog CS@Worcester – Tech. Worth Talking About by jelbirt and used with permission of the author. All other rights reserved by the author.

introduction: software qa and testing

I will now be writing to this blog for my CS-443 course, which will be categorized and tagged for each post for that course. Not too excited to write 8 posts for this, but it is what it is.

From the blog CS@Worcester – V's CompSCi Blog by V and used with permission of the author. All other rights reserved by the author.

CS-443 Introductory blog

This blog is just a setup for CS@Worcester, CS-443

From the blog CS@Worcester – Kadriu's Blog by Arber Kadriu and used with permission of the author. All other rights reserved by the author.

Testing in Python and Sea

Something that is worth noting is Python’s assert keyword. Python has a built in way to create basic unit tests beyond simply printing and making comparisons. This has me thinking about my Sea programming language.

I think much of the hassle I’ve had with unit testing has come from simply getting the environment set up in the first place. If unit tests could be more like a data structure or design pattern – something you know the design of and you can simply implement – I think that would simplify a lot. So while I could create a Sea library for unit testing similar to numerous other libraries (or wait for someone else to make one in Sea), I’d prefer to create a more internal solution.

Sea is fundamentally C, so that means it doesn’t have a lot of high level features that can make unit testing easier. I mean, Sea isn’t even object oriented. That said, the entire design philosophy behind Sea is based on simple syntax that doesn’t add a runtime performance cost. One way I could achieve this is by adding another “stage” to the language. The current design of Sea involves a preprocessor, a lexer, a parser, and a visitor (interpreter, transpiler, or compiler). What I could do is add a tester that would run before the visitor, to test compile-time things (checking types or any other value known at compile time) and then a runtime tester. These could be modular features of the language itself and could be removed if desired.

Another simple solution is to add a handful of keywords to Sea similar to assert. The problem with that is C doesn’t have runtime exceptions. The design would have to be based on the notion that the program is the unit tests. I’ll continue to think this over. After all, Sea is still in its early stages. I’m currently rewriting it (just on the visiting functions now) and then I’ll add functions, pointers, etc. However, if Sea is to become a real, usable language then there will have to be ways of testing code; the simpler and more convenient, the better. After all, that’s the whole point of Sea.

From the blog CS@Worcester – The Introspective Thinker by David MacDonald and used with permission of the author. All other rights reserved by the author.

Converting a Project to Gradle

A few weeks ago, a few classmates of mine asked for some help with converting code to Gradle and I created a quick guide. I’d like to go through and create another guide with more detail.

Assumptions

  • You have Gradle installed.
  • You know which language you are using.

Instructions

The first step is to run gradle init in the project folder that you wish to convert to gradle. I would recommend if you’re unsure about the process or the settings you’ll use, just create an empty folder somewhere and practice creating a gradle project in that. Then, delete it.

Note that depending on the options you choose, you may be asked questions I do not cover here. Just do your best to figure out what it means and use the internet for research. Worst case, you can always modify the settings afterwards.

When you run gradle init, you will be asked some questions about your project. You simply type the number corresponding to the option you want and then press enter (or simply press enter without a number for the default option). In our classes, we always used libraries. If you have a main method, you’ll probably want application.

Next, it will ask you for the language you want to use. This is probably the simplest question. In our classes, we’ve used Java but you can use any of the displayed programming languages.

Next, is the build script DSL. If you don’t have a strong opinion on this, just use the default.

Similarly, next is the testing framework. Use whatever you’re comfortable with. In our classes, we’ve used JUnit Jupiter.

Next you can name the project anything you like.

This last step where it asks for source package for Java is a step where many people mess up. The default option is an undesired package name, since it will include capital letters. I recommend using a lowercase package name.

Now, you can open lib/build.gradle. You can remove the lines and comments for api and implementation. These are auto-generated, but not necessary. Next, if you plan on using GitLab’s ci, modify the testRuntimeOnly line. Notice how the testImplementation line has a version number following a colon. Modify the testRuntimeOnly line so it has the same version as the testImplementation. After these steps, my dependencies looked like this:

dependencies {
// Use JUnit Jupiter API for testing.
testImplementation 'org.junit.jupiter:junit-jupiter-api:5.6.2'

// Use JUnit Jupiter Engine for testing.
testRuntimeOnly 'org.junit.jupiter:junit-jupiter-engine:5.6.2'

}

Now, gradle will have created a gradle/lib, gradle/app, etc folder. Locate your package which will be a subdirectory of gradle/lib/src/main/java in my case. The main folder will contain your source files and the test folder will contain your tests. Move all of your source files into main/PACKAGE_NAME and move all of your tests into test/PACKAGE_NAME. You can delete the files gradle generated for you.Then, if you’re using java, you’ll need to go into each java file and add a package declaration. Make sure its the first line and that there is only one. It should look like:

package PACKAGE_NAME;

Where PACKAGE_NAME is replaced with the package name. You can then run tests with the gradle test command. If you want to have GitLab run your tests for you when you push a new commit, download .gitlab-ci.yml (ideally the newest version so not from this link) and add it to your repository. Now, things should hopefully work.

Closing Comments

It’s taken us a while to become familiar with Gradle so I think it was worth me writing all of this down (albeit hastily) to help other people in the future, and especially to help me remember it if I ever use Gradle again in the future.

From the blog CS@Worcester – The Introspective Thinker by David MacDonald and used with permission of the author. All other rights reserved by the author.

Automated Testing

Automated testing is a great option for repetitive tests such as regression and functional testing but not useable for testing methods such as discovery and usability testing that requires a human to do the work. If used correctly, automated testing can save a lot of time and therefore money. If used incorrectly, it could end up costing more than manual testing.

Certain requirements that the testing process to be automated must meet are that it is repeatable, determinant, and non-opinionated. The first requirement, repeatable, means that it must be ran more than once, or else there would be no reason to automate it. Repeatable tests have three steps: set up the tests with data and environment, execute the function and measure the result, and clean up the data and environment after testing. The second requirement, determinant, means that the outcome will be the same every time the function runs with the same input. For example, when inputted 1 and 2 to an addition function, the outcome 3 must be produced every time it is ran with those inputs. The third requirement, non-opinionated means that there must be a concrete answer. You cannot automate opinionated feedback.

Now that we talked about the requirements of automated testing, now we will discuss the three steps of automated testing: prepare, take action, and report results. In the first step, prepare, you set up the data and test environment where the tests occur. This will require either manipulation of the data or putting the program in a certain state, or both. The second step, take action, the driver will run the tests. This will happen by calling the code directly, or accessing a programs api or user interface. And lastly, report results. The automated system will record and report the results from the test.

When used properly, test automation can save money and time and provide quality test results.

Source:

Testim. (2020, February 03). What is test automation? A simple, clear introduction. Retrieved May 03, 2021, from https://www.testim.io/blog/what-is-test-automation/

From the blog CS@Worcester – Austins CS Site by Austin Engel and used with permission of the author. All other rights reserved by the author.

JaCoCo

After taking a quick look at JaCoCo in class I wanted to learn a bit more about it. This article is an introduction to using JaCoCo and gives some good examples.

The article explains that when you write tests there are some criteria to consider:

“

  • We want to make sure that the parts of the code that are best tested are the parts that are most likely to contain bugs.
  • We want to focus our tests on parts of the application that are critical, the parts where bugs are most likely to lead to a bad outcome for our customers.
  • We don’t want to write tests that repeatedly cover the same areas of the code while ignoring other parts of the code.

“

You can find which parts of the code are the most likely to have bugs by finding the most complex parts. A common way to do this is using cyclomatic complexity (check the article for the algorithm).

JaCoCo measures code coverage, can report on the complexity of each method, and can tell you how much complexity is untested. In the article a lot of the things we covered in class are repeated, but it helps to see it again to reinforce the concepts and also to reference for the future. While we spoke about it in class, a lot of the things that are said are not recorded anywhere.

The article also brings up the use of JaCoCo to measure complexity. After getting as much code coverage as possible you can refactor your code if the complexity score is too high. In the example they reduce a method’s complexity from 21 points to 9 points. That just goes to show how useful JaCoCo is. After reducing complexity and improving code coverage you can be confident your code is ready for a code review.  

View at Medium.com

From the blog CS@Worcester – Half-Cooked Coding by alexmle1999 and used with permission of the author. All other rights reserved by the author.

Static vs. Dynamic Testing

Software testing falls largely into two categories: static and dynamic. Both are valuable, but the scenario around the test being performed determines which method best fits.

Static testing is a testing technique that does not require the code to execute. This can include manual or automated reviews of the code, requirement documents, and document design, all of which are intended to catch errors in the code. Static testing is done in order to prevent errors during the early stages of development and can help locate errors undetected through dynamic testing methods. Reviews are an important facet of static testing and are the primary method for carrying out a static test. A review is a meeting or other process aimed at locating flaws and defects in the design of a program. Apart from physical walkthroughs of the code, there exist tools that automatically search for errors. These tools, such as CheckStyle and SourceMeter, can help developers adhere to standards in code and style.

Dynamic testing is a testing technique that checks the program’s behavior when code is executed. As the name suggests, it is intended to test the dynamic behavior of a software. This encompasses a vast majority of the testing methods we’ve discussed both in this class and on this blog. The two types of dynamic testing are white and black box testing, which are both techniques that I have discussed on this blog before. Dynamic testing ensures that a software is working properly during and after its installation, it makes sure that the software is stable, and it encourages consistency in the software’s functionality.

Sources:
https://www.guru99.com/testing-review.html
https://www.guru99.com/dynamic-testing.html

From the blog CS@Worcester – Ciampa's Computer Science Blog by graceciampa and used with permission of the author. All other rights reserved by the author.

Decision Table – Based Testing

For the fourth assignment in software testing, we had to create a decision table testing. A decision table is a black-box test technique that visually presents combinations of inputs and outputs, where inputs are conditions or cases, and outputs are actions or effects. A full decision table contains all combinations of conditions and actions. Additionally, it shows the causes and effects. Therefore, this technique is also called a cause-effect table. A well-created decision table can help to sort out the right response of the system, depending on the input data, as it should include all conditions. It simplifies designing the logic and thus improves the development and testing of our product. 

The decision table works on input conditions and actions. We create a Table in which the top rows are input conditions, and in the same vein, the bottom rows are resulting actions. Similarly, the columns correspond to unique combinations of these conditions.

Advantages of Decision Table Testing

The system behavior is different for different input, both equivalent partitioning, and boundary value analysis won’t help, but decision table can be used.

It can be easily interpreted and is used for development and business as well.

Help to make effective combinations and better coverage for testing.

Any complex business conditions can be easily turned into decision tables.

In a case we are going for 100% coverage typically when the input combinations are low, this technique can ensure the coverage.

Disadvantages of Decision Table Testing

The number of inputs increases the table will become more complex.

Resources:

https://www.guru99.com/decision-table-testing.html

Decision Table Testing

From the blog CS@Worcester – Tech, Guaranteed by mshkurti and used with permission of the author. All other rights reserved by the author.

Static Testing Tools

As this is the last blog I will be writing for this course, I thought it would appropriate to have it be about one of the final topics that I’ve learned; static testing tools. First of all, what is static testing? It’s a form of software testing that tests the code without needing to execute it. This method of testing is useful for finding and resolving errors during the early stages of the program. This also reduces the amount of potential errors that could appear during later stages of development. There are tools which are used to accomplish this form of unit testing that I’ve had some recent experience with. I’ve learned about a few of these static testing tools, and the ones that I would like to touch upon in this blog are checkstyle and Jacoco.

Checkstyle is one of the first static testing tool that I’ve used, and so I’ll talk about it first. The purpose of this tool is to check that the code being tested follows a specific coding guideline that has been preset. The process of doing so it automatic and so it allows the tester to be able to make sure that the coding standard is properly followed while also being able to save time of checking this themselves. While this tool can improve the quality in the presentation of the code, it can’t detect any coding errors having to do with the logic of the code.

Jacoco is a static testing tool that provides code coverage. It measures line and branch coverage by running various test cases, and displays the result with a visual report that shows which lines of code have been tested and the percentage of the amount of code that executes in each method. This allows the tester to be able to determine which parts of the code need more development.

Static testing tools are great resources for unit testing and can improve the overall development of the program by checking for errors at an early stage. These tools can be used simultaneously to check for multiple issues with a program at once. Checkstyle and Jacoco can both provide information to the tester that allows them to be aware of errors at each stage of the code, which makes the process of development more clear when building the program.

Useful links:
https://www.guru99.com/testing-review.html
https://checkstyle.sourceforge.io/
https://www.blueacornici.com/blog/code-coverage-with-jacoco/

From the blog CS@Worcester – CSBlogger by mjaber54 and used with permission of the author. All other rights reserved by the author.