Category Archives: CS-443

Software Quality Assurance

Yesterday was my final class meeting for CS-443. In it, I was briefly introduced to the concept of Software Quality Assurance (SQA). The presentation on it was mostly focused on Capability Maturity Model Integration (CMMI), which is one set of standards that is often used in SQA to ensure that an organization is capable of developing software well. While I received plenty of information on CMMI from the presentation, I felt that I did not gain a great understanding of SQA in general. For this reason, I decided I would do my own research into SQA to help me prepare for the final exam tomorrow.

I eventually came across this article from softwaretestinghelp.com which I think provides a fantastic introduction to SQA. The article starts of by defining SQA as “a process which assures that all software engineering processes, methods, activities and work items are monitored and comply against the defined standards.” This definition has helped me understand exactly what SQA is much better than the class presentation, which focused on CMMI. In fact, the article’s explanation has clarified to me that SQA can use any set of standards as defined by the organization, and that CMMI is only one possible example of these standards.

While the article emphasizes that SQA is a much broader concept than I previously thought, it also lists several activities and techniques that are generally used in SQA regardless of the selected set of standards. Some of the activities listed include having multiple testing strategies, measuring change impact, and maintaining records and reports. These techniques have all been discussed either in CS-443 or in other classes that I have taken, and seeing them listed here has helped me better understand exactly what happens when SQA is used during a software’s development. The article also discusses several techniques used to enforce SQA during a development process. The main technique that is used is auditing, which involves regular inspection of the products of development to make sure they follow the standards. Other techniques discussed in the article include code inspection, design inspection, and static analysis, all of which I have also been exposed to previously.

This article has made it clear to me that SQA is not simply the enforcement of CMMI, but a much broader concept that can enforce any set of standards using a variety of techniques. Reading this article has helped me better understand what SQA is and what it looks like to use it during development, and I definitely recommend it to anyone who is also having trouble understanding SQA.

From the blog CS@Worcester – Computer Science with Kyle Q by kylequad and used with permission of the author. All other rights reserved by the author.

CMMI and TMMI

After CS-443’s class this week, I decided to further investigate CMMI, Capability Maturity Model Integration, process model. I found testbytes’ article, “What is CMMI? (Capability Maturity Model Integration),” to be illuminating. This article gives an overview of CMMI and why it is used in software development.

At the end of the article is a link to another article by testbytes that caught my attention, “What is TMMI (Test Maturity Model Integration) in Software Testing?” TMMI is essentially CMMI with a focus on testing. While the two are similar, the focus of TMMI makes the implementation of CMMI on software testing smoother. TMMI has the same 5 stages of CMMI with some differences.

Level 1: Initial

  • No test processes in place
  • Failing to meet deadlines

Level 2: Managed

  • Tests are planned and monitored
  • Set test designs

Level 3: Defined

  • All projects are tested from early stages onward
  • Test reviews

Level 4: Measured

  • Properly defined test strategies set in place from beginning of development process
  • Projects tested at every stage to ensure bugless code

Level 5: Optimization

  • Testing practices are structured
  • All processes and outcomes are measured

TMMI is an interesting model that optimizes the testing process and while it is similar to CMMI, I’d still recommend a quick read through this of this article to others.

Articles Referenced:

https://www.testbytes.net/blog/what-is-cmmi/https://www.testbytes.net/blog/test-maturity-model-integration/

From the blog CS@Worcester – D’s Comp Sci Blog by dlivengood and used with permission of the author. All other rights reserved by the author.

A Valuable Lesson for Creating Tests

Article link: https://dzone.com/articles/why-you-should-stop-now-testing-every-class

A great article I read today goes over the importance of creating the correct kinds of tests for software. In this article, the author thoroughly explains the importance of testing the “behavior” of software versus “implementation details” and how not “every single class” needs to be tested. I think the author does a great job with both an easy to understand narrative and tons of example code that illustrate the importance of testing that software performs to specification versus testing how the code is written. The example code given in the article goes further with linking to an entire GitHub repository for the example code (which elaborates beyond what is shown in the article) which I really appreciate as it lets me see exactly how this concept aligns with a complete program. The most interesting part of this article for me was in the author’s explanation of how refactoring code can cause tests that are written (incorrectly) to a specific implementation to fail. I think it is important to see this point that putting in extra effort in developing software but on the wrong things can actually do more harm than good further down the line. Especially as the author points out if you were to further refactor the code and change how it works (even return types as the author demonstrates!), this causes bad tests to fail, which only creates even more work. I never really thought of how refactoring could create more work in a testing scenario before reading this, but I like how refactoring further helps the overall program by exposing the appearance of useless tests. One other part of the article I really enjoyed was the side narrative the author gave while working through his example of how the steps they took followed TDD (test-driven development). This is also something I wanted to learn more about, and this article also provides a good example of TDD by explaining how certain practices taken in the code example adhere to the rules of this methodology.

Overall, I think this article covered an important topic well, and serves as a great reminder at the end of our testing course to make sure we are testing the right things in our software and that we are not creating too many, and useless tests that only hinder our efforts and not help us. I will certainly keep this article in mind when writing future tests to ensure I’m not wasting my time by creating the wrong kinds of tests or making sure every line is tested in a program.

From the blog CS@Worcester – Chris' Computer Science Blog by cradkowski and used with permission of the author. All other rights reserved by the author.

Mutation Testing

Last Thursday in CS-443, I was introduced to Mutation Testing. This is a testing method which uses tools to embed faults into a project’s code before running its tests. The idea behind it is that, if the tests are written well, then they should fail when mutations are introduced. The in-class activity introduced Mutation Testing by having us clone a Java project and running the PIT tool to test the project with mutations. While the activity effectively demonstrated how to use PIT, I felt that it did not give me a complete understanding of Mutation Testing in general. This was mainly due to some confusion I had while trying to figure out how to run PIT. For this reason, I decided to research Mutation Testing on my own to help improve my understanding of it. I eventually came across an article from softwaretestinghelp.com that I think provides a great introduction to Mutation Testing.

Link to the Article:

Mutation Testing: Testing Technique with a Simple Example

The article demonstrates Mutation Testing with an extremely simple pseudocode example consisting of a single if statement. The example has five different types of mutations applied to it (called ‘Mutation Operators’) which demonstrate how adding mutations can alter the results of tests. Some of these mutation operators, like relational and arithmetic operator replacement, were demonstrated by the class activity. Others, like statement removal and absolute value insertion, were completely new to me. This taught me that there is a much wider variety of mutation operators than the activity demonstrated.

I also found that the article made it easier to understand how the mutations affected the results of the tests. In this article, the results of the Mutation Test are listed in a table with each column representing a different mutation made to the example code. This makes it possible to see exactly which test values either kill the mutants or cause them to survive. It also makes it easier to notice syntactically incorrect, equivalent, and trivial mutants, which the article mentions should be avoided. I disliked how the Mutation Test results were reported by PIT as a generic list of mutations with vague information describing why they survived or were killed. Having the results in the article be presented in a table and being able to refer back to the actual mutated code made it much easier to understand the results of the Mutation Tests. Compared to the in-class activity on PIT, this article uses a much simpler example to demonstrate mutations and presents the results of its Mutation Tests in a much clearer way. This has helped me better understand the fundamentals of Mutation Testing, and I think I will be able to use tools like PIT more effectively as a result.

From the blog CS@Worcester – Computer Science with Kyle Q by kylequad and used with permission of the author. All other rights reserved by the author.

Mockito

Mockito is a framework in Java that creates Test Doubles for users that can be used with Junit. Test Doubles as described in my last blog are basically simplified tests for your code that more or less test setup over functionality. They are used to help you make sure you are setting up your code correctly in the early stages before it gets overly complicated. It’s good practice to test at this early stage in the process because simple setup and organization errors can become an overwhelming problem further down the line if you don’t find them early on. Mockito is a great framework to help you quickly generate these Test Doubles. Mockito creates mock objects with dummy values to make sure tests pass, as long as your setup is correct.

Check out this code example:

package com.journaldev.mockito;

import static org.junit.jupiter.api.Assertions.assertEquals;

import static org.mockito.Mockito.when;

import org.junit.jupiter.api.Test;

import org.mockito.Mockito;

import com.journaldev.AddService;

import com.journaldev.CalcService;

public class CalcService1Test {

@Test

void testCalc() {

System.out.println(“**— Test testCalc executed —**”);

AddService addService;

CalcService calcService;

addService = Mockito.mock(AddService.class);

calcService = new CalcService(addService);

int num1 = 11;

int num2 = 12;

int expected = 23;

when(addService.add(num1, num2)).thenReturn(expected);

int actual = calcService.calc(num1, num2);

assertEquals(expected, actual);

}

}

This code uses a mock() method.

To continue reading about this example check out this website:
https://www.journaldev.com/21816/mockito-tutorial

From the blog cs@worcester – Zac's Blog by zloureiro and used with permission of the author. All other rights reserved by the author.

Static testing and Dynamic testing

Static testing and Dynamic testing is two different approaches to testing available for developers and testers in the software development process. In order to get the most out of these tests, this has to be chosen carefully and it is important to understand the benefits and the limitations of each one.

Static testing is a test method where code are not being executed and it can be done manually or using a set of tools. This type of testing would check the syntax, required documentation and design of the code. Static testing also includes security testing to analyze the software for potential errors, code flaws, or vulnerabilities. This method can be start in the early development stage of the program, and it can be done on work documents like specification documents, design documents, web page contents, etc. Static testing techniques include:
–  Inspection: The main purpose of this is to find defections. This task can be reviewing the check lists, work documents, or code walkthroughs and it is conducted by moderators.
– Walkthroughs: This technique would require the author to conduct a meeting to explain the product. Participant can ask questions and a scribe is assign to make notes.
– Technical review: This technique is to check if the written code is matched with technical specifications and standards. Test plans and strategy is reviewed here.
– Informal review: Documents in this technique should be reviewed informally and comments should be added.

Dynamic testing is done when code is in execution in a runtime environment. When code is running, the predefined input should get the expected result for the test to return positive. With this, developers can observe features of the program and monitoring the effect of this program on the running hardware. There are two types of Dynamic testing: functional and non-functional testing. Dynamic testing techniques include:
– Unit testing: testing individual modules by developers.
– Integration testing: testing the performance when different modules run with each other.
– System testing: testing the system as a whole.
– Acceptance testing: testing from user’s perspective end.

Article can be found here.

From the blog #Khoa'sCSBlog by and used with permission of the author. All other rights reserved by the author.

Black Box, White Box and Grey Box testing

Black Box, White Box and Grey Box are there most common terms in testing as it is really important. These terms are selections of tests that developers have that is based on their purpose of testing, what is being tested, and they determine which what tools or technologies to be used in order to tackle to problem efficiently.

Firstly, let’s talk about Black box testing. This type of test is really common on testing user interface. It treats the program as if it is a “black box”, or refer to testing without knowing or be able to change the internal implementation. It is also known among developers as closed or opaque box. The advantage of this type of testing is pretty huge, as developers can focus more on how to test the feature, rather than having to get to know the internal code. It also needs low time to prepare, and designed to simulate the perspective of users. With those potential advantages, the trade off of Black Box testing is also huge, as it might have a chance to have redundancy in tests, and QA engineer has to blindly explore the system without knowing the source of error, if the program has some.

White Box testing is basically opposite to Black Box testing, which allows the QA engineer to fully access to internal data structures and implementation. They are not only able to view the source code, they can also manipulate the code as part of the testing process. This is also known as clear box or open box testing. The advantages could bring a lot of benefits to developers as codes can be tested during the development process of a feature, which allows them to spot out bugs almost immediately. With the ability to access to the internal, QA engineer can use test in order to test for performance and optimization, and also it is a good practice to have code reviewed by multiple developers. The disadvantages can also be huge in contrast to what are mentioned before. White Box testing would require very in depth knowledge about the system and the duration of these test should correspond to the length of the source code, which is a lot. This type of test can also have a big influence from tools from development process that can affects the test cases and it is not scalable in anyway.

And finally, Grey Box testing, which is a combination of Black Box testing, and White Box testing. It allows the QA engineer to see the internal implementation, but cannot modify them directly. The advantages and disadvantages from this, is pretty much also combine between the two previous type of test. Grey Box testing can have the program to be seen as the level of user, while also can use the internal implementation to optimize the test cases. The downside of this is that QA engineer will not be able to change the source code, and have to play around with it multiple times to find a correct way to tackle the problem.

Article can be found here.

From the blog #Khoa'sCSBlog by and used with permission of the author. All other rights reserved by the author.

Mutation Testing

PIT mutation testing is great way to test code that may seem like it passes everything, but still contains errors. Mutation testing will insert these faults into your code and will then retest the code to make sure they are properly handled. If a mutated test is killed, that means it was properly accounted for by the test cases and code. If a mutation survives that means that somewhere in the code, there is an error that is not being handled properly. An example of mutation testing will be something along the lines of taking a less than sign ( < ) and making it into a less than or equals ( <= ) to see if the code will be able to kill this mutation or if it survives then something must be fixed.

From the blog CS@Worcester – Journey Through Technology by krothermich and used with permission of the author. All other rights reserved by the author.

Test Automation

Article link: https://dzone.com/articles/starting-automation-testing-from-scratch-here-is-w

Throughout this semester we have learned about how to create tests in a mostly “manual” way, today I want to explore an article that serves as an introduction and overall guide to automation testing. I especially wanted to learn about this topic since it has repeatedly come up in other articles I’ve read on this website.

Overall, I found the content of this article to be great and I think it thoroughly covers every aspect I can think of, and acts as a great beginner’s guide to this topic of automation testing. The author methodically covers everything on automation testing from the benefits of switching, to figuring out which areas of your testing should be automated.

Looking at the section of this article that lists the different areas automation testing can help is particularly interesting to me as most of the types of testing here I am unfamiliar with (and aren’t covered by our testing course) but would like to know more about. This includes areas such as regression testing, which I just learned about, and performance testing. From this article it seems that automation testing seems to help particularly with cross-browser testing. This made me think about how different web browsers are tested which is something I have only done by manually checking that the application functions properly by going through the program as a user. In retrospective it makes sense that some automated tool can make this much easier and I would like to see how this works more.

I think one of the most important points the author makes (and states many times) is “do not aim for a 100% automation”, which makes sense as the author explains how it is not necessarily feasible or beneficial to have all of your tests automated. Furthering this point, the other interesting part of this article is how the author explains that manually testing is better for certain aspects of software. After reading these reasons, I completely agree with the author that certain areas such as testing the actual user experience of a program makes more sense to test manually instead of automating. Another aspect I really like about this article that I like is that the author continues throughout the article to focus on the practicality of automating testing from a business standpoint and takes into account the amount of investment and return on investment that can be gained by automated testing.

This article has certainly made me more interested in testing automation and I would like to see how tools implement this, especially in areas the article says can really benefit from this such as regression or browser testing.

From the blog CS@Worcester – Chris&#039; Computer Science Blog by cradkowski and used with permission of the author. All other rights reserved by the author.

A Retrospective and Review of Unit Testing

Article link: https://dzone.com/articles/why-do-programmers-fail-to-write-good-unit-tests

In this blog post I wanted to examine an article I found about unit testing. I think that since unit tests were one of the first types of testing we learned about and we have used unit tests as a foundation for many other testing methods this semester, reviewing this article serves as a great retrospective (and provides another viewpoint) on the importance of unit testing and defining criteria on writing “good” unit tests.

In the article, the author defines the significance of unit tests in creating software. They then go on to define some various qualities that make for a “good” unit test. I agree with the listed points, but I think this article could benefit by providing a few examples of good unit tests and maybe a few bad ones for comparison, especially since this is an article about programmers not correctly writing unit tests. I also think the article could better define what these “units” that are being tested are, since this term is used frequently throughout the article. Additionally, I think the article should better define what specifically is being tested in these “units” besides just saying “examine and test each unit of code separately”. Again, this is where some examples would greatly help illustrate the author’s point. The article then goes to list many benefits of writing unit tests. It ends with an interesting (and great) point of how just having unit tests isn’t necessarily beneficial (and can even be detrimental) if they are poorly written.

Overall, I think that after learning about and using unit tests a lot this semester, that this article can serve as a good introduction to the topic, and it certainly reaffirms what I have learned about unit testing, but I also think it requires some revisions to be more helpful in demonstrating the points it makes, as I find it to be somewhat vague in the fundamental parts of presenting the main concept of unit testing. I think that having a well written article about this topic is especially important as this type of testing is used as the basis for many other types of testing we have learned about. I also think that this article could include this point of how unit testing is used in the larger contexts of other types of testing such as ensuring code coverage to further prove its importance.

From the blog CS@Worcester – Chris&#039; Computer Science Blog by cradkowski and used with permission of the author. All other rights reserved by the author.