Category Archives: CS-443

Automation Testing

For the last and final blog for the CS 433 class, I want to focus on an interesting topic that is automation testing. Nowadays, most companies have implemented automated testing environments. It is the best way to increase effectiveness, test coverage, and the execution speed of software testing. Before I get started, what exactly is automation testing? Well, automation testing or Test automation is a software testing technique that performs using various software tools to execute a test case suite. The automation testing software can enter test data into the system and compare expected and actual results with the generated details test reports. There several benefits of using test automation, it is the best way to increase effectiveness and save money. Other factors include manual testing, which is time-consuming, Test automation does not require human intervention and can run automated tests unattended or even overnight, it is reliable in results, improve accuracy, better speed, and lastly, you can test frequently and thoroughly.

Now let us talk about the different types of test cases that use in automation. While researching a few of them once that I found are High-Risk Business test cases, Test cases that are repeatedly executed, Test Cases that are very difficult to perform manually, and finally, Test Cases that are time-consuming. The way automation testing works is using the pyramid strategy which consists of three levels. According to the blog, the base represents unit testing, then the integration/service layer, the next API testing, and finally the GUI test sits at the top of the pyramid. There are many types of automated testing some of which we learned in class, for example, unit testing, black box testing, regression testing, and even data-driven testing. Automated testing is becoming a popular strategy among many companies, I believe that majority of the companies will implement fully on this technology.

https://www.guru99.com/automation-testing.html

https://smartbear.com/learn/automated-testing/what-is-automated-testing/

From the blog Derin's CS Journey by and used with permission of the author. All other rights reserved by the author.

Test Doubles

Unit testing is a form of software testing where part of the software is tested and working. Recently I’ve been introduced to some methods of unit testing where a temporary object is created in place of an existing object in order to perform testing. The term for this is “Test Doubles”, as just like an actor in a movie has a double, doubles are created for objects in the program for the purpose of testing them. I’ve learned about different types of test doubles in my Software Quality and Test Assurance course, and I’d like to examine a few and give a brief explanation about how they work. The names of these test doubles are dummies, stubs, fakes, and mocks.

Dummy objects are made for the purpose of allowing methods to call an argument, and therefore they do not have any implementation themselves. These are used for methods are not actually important for your tests. For example, if you are trying to test a method and have it return something specific, then a dummy method can be created instead of making a method with code that isn’t relevant for the test.

Stubs, similar to dummy objects, are created for the use of testing code without needing to fully develop it. A test class is called to the stub, and the stub returns a value that is controlled. This is useful for scenarios where the result of the test changes depending on what the input is.

These test doubles make testing for code easier. They reduce the need of excessive code and allow for the methods in the test class to simply call the these test doubles instead. Each different test double has its own use depending on the objective you are trying to reach with your testing.

This article and blog provided very useful info regarding what each test double does and demonstrated each type of test double with examples.
https://dzone.com/articles/test-doubles-mockito
https://mokacoding.com/blog/swift-test-doubles/#:~:text=Dummy,the%20behaviour%20you%20are%20testing.

From the blog CS@Worcester – CSBlogger by mjaber54 and used with permission of the author. All other rights reserved by the author.

Blog Post #5 Static Testing

After working with Checkstyle in the recent activity, I wanted to look more into other static testing tools, but this led to me focusing on the purpose and function of static testing. I had an idea on how it worked prior to the activity, got practice using one of its tools in the activity, but I wanted to read more about static testing in general to get a better grasp on the concept and how it works.

Static testing is based around testing code without actually running the code, as opposed to dynamic testing that checks the code when it runs. The two main static testing techniques are manual examination and automated analysis using tools. Static testing is effective because it can catch early mistakes, takes less time to test, and leads to improved productivity. Checkstyle was the main tool that we worked with, but there are others such as Soot and SourceMeter.

I think the main aspect of static testing that I wanted to learn more about was how it compared to the normal testing that we have been working with, and after working with stubs and mocking in the past activities, it feels similar to mock testing because it seems more about testing behavior than state. It felt like that way to me because JUnit tests can still run when the static test picks up errors if the errors are programmed to be ignored, similar to how a JUnit test may pass but not work properly because it picked up a different method that had the same return type, which is the purpose of behavior testing.

However, with static testing, the proper method can be called and the code could have the right behavior and state, but still not pass because of errors found with the test, such as Checkstyle, which finds errors due to the selected coding style/standard. These errors can also be coded to ignore and still run.

https://www.guru99.com/testing-review.html

https://www.guru99.com/static-dynamic-testing.html

From the blog Jeffery Neal's Blog by jneal44 and used with permission of the author. All other rights reserved by the author.

Static Analysis Tools

For my final blog post for the class, I decided to look at a blog post from Codacy. In this blog post, they talk about how only about 15% of developers use static analysis tools and that more people should. The blog post starts by talking about in the past a lot of developers manually look for errors and bugs within their program and how this method of code analysis is very error-prone because humans are prone to making errors and are likely to miss a bug or two. I hate to admit it but I agree with them on this point. Human beings are flawed and will inevitably make a couple of mistakes. We don’t know when or how but we will. This is just a part of human nature. Computers can run many more tests than we can and very quickly find errors and save a lot of time and money in development. Even though static analysis tools are powerful, we still need to pair with some dynamic testing because static analysis cannot account for every thing that could happen or go wrong.

What are Static Analysis Tools?

From the blog CS@Worcester – Just a Guy Passing By by Eric Nguyen and used with permission of the author. All other rights reserved by the author.

Testing Functional Code

In programming, it is essential that developers can ensure that their code works as expected. One way to ensure this is unit testing. However, unit testing and many of the general testing styles are only convenient and simple for basic types of projects. For example, unit tests are great for public libraries made up of classes and functions that can easily be tested. Not only is it easy to test these, but it is simple to understand. You just make objects and call methods and check the results. However, not all code is made up in that fashion.

A clear example of this is a programming language such as my Sea language. In theory, I could take a few weeks and create proper tests for everything involved. However, creating production-level unit tests for a project of this size by yourself is tedious and time consuming, to say the least. One way I find convenient to help test my language is to simply create a sea file with code I want to test the interpretation/transpilation of and then run both the transpiler and interpreter to check the results. Another thing I did was add a debug option to print out the generated tokens, AST, and memory.

This is admittedly janky. There is no standardization for my testing process. Luckily, most of the problems I run into either generate a Python exception, or an obviously incorrect output. That said, maybe that’s only because those kinds of problems are the only ones I find and there are tons of hidden errors that are laying quietly. That overall is one of the best things about Python, and my motivation for Sea. In Python, it is really easy to write thirty lines of code and then run it and have it execute without any errors. This is because the syntax is so simple and readable that its easy to find errors before you even run the code.

Sometimes, I think its alright to not properly commit to testing code. Code that relies more on user input than parameter input (what I’ve called functional code) can be incredibly challenging to properly test. For instance, if I were to properly test Sea, I would need to create sea files that have almost every combination of valid syntax. Not only that, but I’d need combinations of valid and invalid syntax as well to make sure the code finds the errors. The point of testing is to save time and to ensure valid code. In the case of a programming language, I think its easy enough to ensure valid code and in exchange you can save almost half of the total time you’d otherwise need to be spending.

From the blog CS@Worcester – The Introspective Thinker by David MacDonald and used with permission of the author. All other rights reserved by the author.

Starting off Clean

I have a noticeable tendency when I’m coding or even when I’m gaming. After a while of making progress, I often reach a point where I’ve learned enough to remake – whatever it is that I made – much better. So, I’ll get to work on modifying what I currently have in an attempt to improve it. This can work sometimes; usually, however, I end up with a lot of complexity laid out in front of me and I become lost in it. Then, I follow my tendency to simply start over.

In my Sea programming language for instance, I started without any real knowledge of how to make a language. I followed a tutorial with pretty bad code and modified it to serve my needs, learning along the way. I’ve recently added all of the combined assignment operators as well as loops. The next step is to add a method of mimicking main memory so I can add functions and proper variable scopes, as well as memory management. So, I want to start off by refactoring the code I currently have. I made decent progress but then I found myself with numerous files open without any new working code. Since I was rewriting basic features anyway, I finally gave in and started rewriting all of the Python code from scratch. The new code can currently be seen in the overhaul branch, until I finish and merge it. (If you’re reading this in the future, that link might not be useful anymore so try this compare link.)

So, I’d like to discuss the benefits and drawbacks of this habit of mine. First off, sometimes, starting off clean just helps manage the complexity of a problem. Being able to go through every file of code and determining which lines are good is incredibly useful and you can find things you otherwise won’t notice. With git and the ability to copy and paste, it takes hardly any time to reincorporate good code. However, I tend to still think of this process as something I ideally shouldn’t do. Maybe it would save me a lot of time to simply refactor what I need to. It’s possible I just need to get better at refactoring. However, it’s also possible that I’m simply new at a lot of this and in the future I’ll write more solid code from the beginning anyway.

I think, at least in this current Sea overhaul, starting off clean has been nothing but positive for me. This is code I’m very new with and having added so many features, I understand it significantly better than when I first wrote it. I think at the end of the day, both refactoring and starting off from the beginning are valid strategies. As we learn, we’ll learn which to use when. If you only need to make a few changes, then just refactor the code you already have. However, when you’re faced with a massive redesign, just take the time to rewrite it. The problem isn’t so much which you choose to do, but rather when you decide to do one over the other.

From the blog CS@Worcester – The Introspective Thinker by David MacDonald and used with permission of the author. All other rights reserved by the author.

Testing with Mockito

During class, we have been doing many testing methods including Junit 5 and for the past two weeks we have been using Mockito, so for today’s blog, I want to focus more on the framework and testing with Mockito. Before we get started let us talk about what is mocking. Mocking is a process of developing the objects that act as the mock or clone of the real objects. In another word, mocking is a technique where mock objects are used instead of real objects. The purpose is to isolate and focus on the code being tested. Mock uses objects such as Fakes, Stubs, and mocks. A fake object has working implementation but takes shortcuts which makes them not suitable for production. Stub object usually does not respond to anything outside that is programmed in for the test. Mocks are objects that are preprogrammed with expectations which form a specification of the call that is called to receive.

Now let us move on to what Mockito is, Mockito is a java based mocking framework that internally uses the Java Reflection API to generate mock objects for a specific interface. Mock objects as I explained previously are used as dummy objects. The main purpose of using the Mockito framework is to simplify the development by using mocking external dependencies and use them in the test code. Few benefits of Mockito include writing beautiful tests with a clean & simple API, it creates mock objects using annotations like for example @Mock, and In Mockito, there is no requirement for writing your mock objects. The first link I have provided gives a lot of detailed examples of Mockito. It has helped me to learn the framework fully. I highly recommend to check it out and follow the example.

https://www.vogella.com/tutorials/Mockito/article.html

https://site.mockito.org/

https://www.javatpoint.com/mockito

From the blog Derin's CS Journey by and used with permission of the author. All other rights reserved by the author.

Object-Oriented Testing

Some of the common methods of testing we know and learned are unit testing, which tests the software at a single unit or class file, and we also have system testing that tests different part of the system. However, as software is becoming more complicated and there are many class files in a software project, one important method of testing that has become useful over the years is object-oriented testing. Normal testing methods won’t be able to be used in these sorts of circumstances because of the different object-oriented features such as inheritance, polymorphism, abstract classes, and other related concepts to it. There are different ways to test the classes and the same methods for testing functions won’t be able to be applied because a single function is completely different than an entire class output.

For object-oriented testing, there are a lot more dependencies that are required compared to normal testing styles. Testing functions are a lot more simpler than testing objects and so more factors need to be taken into account. The first additional testing method for object oriented testing is known as fault based testing. This way of testing tries to come up with test cases that predict any faults that are possible and these test cases are there to solve the errors. The fault based testing doesn’t find all the possible errors and it only tests for the requirements of the client. The next type of testing method is class testing based on method testing and for this one each method is tested similar to unit testing and all the methods are tested for a class one time at the bare minimum. The next type is random testing is for random test cases that is responsible for the operations that are related to behavior of certain categories of the software. The next version of testing for object-oriented testing is partition testing which takes into account inputs and outputs for a certain category and this reduced the total number of test cases. The last contrasting test case is scenario-based testing which takes into account different actions that a user will take and tests for those separate scenarios. Overall, object-oriented testing has similar concepts to conventional testing but there are many other types of testing that are also included that vary from the normal way.

Links:

https://www.geeksforgeeks.org/object-oriented-testing-in-software-testing/

From the blog CS@Worcester – Roller Coaster Coding Journey by fbaig34 and used with permission of the author. All other rights reserved by the author.

Detailed and basic usage of Mockito

 Mockito is a Mock framework for Java single-testing, but it can also be used with other single-testing frameworks in addition to JUnit. Mockito changes the behavior of a class or object, allowing us to focus more on testing the code logic without the effort of constructing the data.

The basic concept

Mocks can be of two types, Class and Partial，so Mockito is called spy. The behavior of changing methods on mock objects is called Stub.

A Mock process is called a Mock Session, and it records all the Stubbing. It consists of three steps:

+———-+ +——+ +——–+ | Mock/Spy | ===> | Stub | ===> | Verify | +———-+ +——+ +——–+

Class Mock

A Class Mock changes the behavior of a Class so that the object it mocks completely loses its original behavior.

Method returns default values (null, false, 0, etc.) if it is not pegged.

The most basic usage is as follows:

1
2
3
4
5
6
7
8
9
10
11
12
 
import static org.mockito.Mockito.*;

// use List.class to creat a mock subject --- mockedList


List mockedList = mock(List.class);



//operation of mockedList


mockedList.add("one");
mockedList.clear();



//validation


verify(mockedList).add("one");
verify(mockedList).clear();

Partial Mock（spy）

If we only want to change the behavior of an instance, we need to use spy:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
 
List list = new LinkedList();
List spy = spy(list);

// optionally, you can stub out some methods:
when(spy.size()).thenReturn(100);

// using the spy calls *real* methods
spy.add("one");
spy.add("two");

// prints "one" - the first element of a list
System.out.println(spy.get(0));

// size() method was stubbed - 100 is printed
System.out.println(spy.size());

// optionally, you can verify
verify(spy).add("one");
verify(spy).add("two");

As you can see from the code, the main difference between a Spy and a MockSettings is that the MockSettings for a Spy needs to be passed in a SpiedInstance.

The default Answer to a spy is CALLS_REAL_METHODS, which means that if a method is not stub, it performs its real behavior.

The default Answer to a mock is RETURNS_DEFAULTS. Methods that are not stub return a default value.

source：

https://www.vogella.com/tutorials/Mockito/article.html

https://howtodoinjava.com/mockito/junit-mockito-example/

From the blog haorusong by and used with permission of the author. All other rights reserved by the author.

Software QA Log #5: Learning and Utilizing Mockito

In a previous post I mentioned mocks in passing as one of the techniques used for test doubles. However, in that brief mention, I noted that mocks can be utilized in unit testing without having to create an actual object with actual values to test the behavior of a class. In essence, mocks aren’t too dissimilar to fakes or stubs; mocks are also pre-programmed to follow certain behaviors. However by utilizing mocks , unlike those test double techniques I mentioned in my previous post, programmers can utilize mocking testing frameworks, which can help significantly in automating and conducting unit testing. Though there are multiple mocking frameworks that can be utilized, I will be focusing mostly on Mockito as it is more well-known, however there are multiple other frameworks that may be stronger where Mockito is weak.

First, it is important to understand what a mock object is in order to understand how it can be used with frameworks like Mockito. Mock objects are essentially replicas of actual objects that are to be used in their place during behavior testing. By utilizing interfaces, a mock object can replicate the behavior of an original object without needing to address any other dependencies. Thus, mock testing allows us to test parts of a program without needing to use its resources themselves. One example where we would want to use mocks would be if our program needed to connect to a database and retrieve information. Instead of having to actually access the database and use real data, the mock object adjusts its behavior to accommodate for the specific unit test. In essence, mocks are similar to fakes.

An example of a mock is the following:

Source: https://dzone.com/articles/mockito-pros-cons-and-best

Unlike fakes, mocks can utilize multiple mocking frameworks. One such mocking framework is, as I mentioned earlier, Mockito. Mockito is a widely used JAVA-based mocking framework that can be often used in tandem with JUnit. Unlike JUnit, which is used to test the source code, Mockito uses mock objects to test in place of real objects. Moreover, unlike JUnit, Mockito can help in keeping track of when and how many times a method is being invoked and even verify if a certain method is being called, something that cannot be done right away with JUnit. However, along with the good things that Mockito brings to the table, Mockito has its own weaknesses, namely that it may not mock constructor methods as currently as version 3.x, or it cannot verify methods like toString().

Like with other test double techniques, mocking and Mockito are not the end-all be-all of testing, but some tools that can be utilized to help create and automate reliable tests for software. Depending on the software’s and the programmers’ needs, different frameworks or techniques might be preferable, however mocking and Mockito are among the solutions that one can learn and use.

Sources/Articles referenced:
1) https://www.toptal.com/java/a-guide-to-everyday-mockito
2) https://circleci.com/blog/how-to-test-software-part-i-mocking-stubbing-and-contract-testing/
3) https://www.accenture.com/us-en/blogs/software-engineering-blog/to-mock-or-not-to-mock-is-that-even-a-question
4) https://www.diffblue.com/blog/testing/mocking-best-practices/
5) https://swiftsenpai.com/testing/test-doubles-in-swift/
6) https://github.com/mockito/mockito/wiki/FAQ

From the blog CS@Worcester – CompSci Log by sohoda and used with permission of the author. All other rights reserved by the author.