For the fourth assignment in the Software Quality Assur&Test class we had to create a program graphs and a DD-Path graph. We had worked during class time in different activities that covered this assignment so I would say I really enjoyed working in this assignment. We had to complete three parts each with a different level of difficulty, but I did only basic and intermediate. I learned to understand more of the way how to create the graphs and what you need to place in the graph and what not.
We have given a java code and I have to create the Program Graph for the play () method. In a Graph, we have a set of nodes (a.k.a vertices) and these nodes relate to each other with the help of some edges. The nodes or vertices are used to store data and this data can be used further. To build the graph is easy but complicated at the same time. All the 27 lines of the code are represented with 27 nodes and some of the nodes have more than one line.
A graph is a data structure that consists of the following two components: 1. A finite set of vertices also called as nodes. 2. A finite set of ordered pair of the form (u, v) called as edge. In a Connected Graph, from each node, there is a path to all the other nodes i.e., from each node, you can access all the other nodes of the graph. But in a Disconnected Graph, you can’t access all the nodes from a particular node. Our graph is a combined one.
After finishing the graph, I had to Create the DD-Path Graph for the play () method given above and include the table translating from Program Graph nodes (from your graph above), with the corresponding DD-Path node labeled with a letter. The most difficult part is done in the first part so creating the DD-path and the table it didn’t took me a long time to translate it to what was required. Overall, it was fun assignment and I really enjoyed.
This past semester I have been spending a lot of time in my QA class working with JUnit 5. I want to be able to take my familiarity with JUnit and apply the same principles with unit testing in Python. I am leaning toward a data-centric career path and Python is widely used for data analytics, so this would be valuable information for me.
This post is not an expert-authored tutorial on Python unit testing because I, myself, am just getting started with it. In this post I will instead give tiny, bite-sized examples of just the basics, translating it from JUnit to Python unittest. I built identical, small classes in Java and Python, and will build tests to go with them. Below are the classes in Java and Python, respectively.
/**
* Simple class with basic methods, written in Java
* @author Christian Shadis
*/
public class main {
public static void main(String[] args){
int i = 0; // dummy code to keep compiler happy
}
public static int addTwoNumbers(int x, int y){
return x + y;
}
public static String toCapital(String str){
return str.toUpperCase();
}
}
# Simple class with basic functions, written in Python
# Author: Christian Shadis
class main:
def add_two_numbers(x, y):
return x+y
def to_capital(string):
return string.upper()
Writing the unit tests in JUnit is simple: we import the JUnit assertions, and the @Test annotation. Then we create the test class, each of the two tests, setup, exercise, and verify just as always.
/**
* Test Class for main.java
* @author: Christian Shadis
*/
import static org.junit.jupiter.api.Assertions.*;
import org.junit.jupiter.api.Test;
public class maintest {
@Test
void testAddTwoNumbers(){
int result; // setup
result = main.addTwoNumbers(566, 42); // exercise
assertEquals(result, 608); // verify
}
@Test
void testToCapital(){
String result; // setup
result = main.toCapital("string"); // exercise
assertEquals(result, "STRING"); // verify
}
}
Luckily, writing unit tests is just as easy in Python as Java. We import the unittest library and the other class, define the class, and add a Test Case object as the parameter.
import unittest
from main import *
class maintest(unittest.TestCase):
import unittest
from main import *
class maintest(unittest.TestCase):
def test_add_two_numbers(self):
pass
def test_to_capital(self):
pass
Now we need a test case for each of our two functions, add_two_numbers and to_capital. Python’s unittest objects have very similar assertions as in JUnit. Use assertEqual(x, y) to check that x == y. This is the assertion we will use in this example, but any of the following are commonly used unittest assertions:
assertEqual
assertNotEqual
assertTrue
assertFalse
assertIs
assertIsNot
assertIsNone
assertIsNotNone
assertIn
assertNotIn
assertIsInstance
assertIsNotInstance
assertEqual takes two arguments and, as the name suggests, asserts their equality. See the implementation below:
def test_add_two_numbers(self):
result = main.add_two_numbers(33, 44)
self.assertEqual(result, 77)
def test_to_capital(self):
result = main.to_capital("hi")
self.assertEqual(result, "HI")
Run the tests and you will see them both pass. Below is a screenshot of the tests executing in Python and then in JUnit. As you can see, the Python tests are slightly faster than the Java tests, but not by much. I used IntelliJ IDEA and Pycharm IDE.
I have found the most helpful way to learn testing is to just play around with the unit tests, see what works and what doesn’t, see what causes failures and what those failures look like, and so forth. I would suggest any other beginner QA student to do the same. Playing around with different assertions and looking at the unittest documentation is a great way to learn this library. I hope this post gave you some insight on how to get started with unit testing your Python modules if you have done some work with JUnit in the past.
For my fifth and final topic review for Software Quality Assurance and Testing, I decided to go over test automation. I found a lot of good sources on the internet to help me understand what it is. A lot of the testing practices that we have learned about this semester have been manual testing rather than automated testing. This just means that we would write the tests and go through them ourselves rather than let the computer do it by itself. This works really well with testing methods like discovery testing and usability testing. The reason we would use automation testing is because it is much easier to have the computer run tests for us (especially in cases in which we have repetitive testing). We do not want to be wasting our time running repetitive tests over and over again when the computer could do it all much faster and with a lot less effort. Test automation’s main appeal is the speed that comes with it. One would think that we should just use this type of testing all the time, but the reason we do not is because some testing methods have to be done manually. One of the best sources I found was on testim.io, and I will make sure to add the link for that at the bottom of this post. The article is very good because it describes what automation testing is, what its criteria are, what types of tests are classified as automation, and the general process of using this testing method.
Seeing as we are continuing our learning with Mockito, specifically going into spies in Mockito, figured it would be worthwhile to do some research into them. I found this article (https://www.baeldung.com/mockito-spy) from Baeldung by Eugen Paraschiv detailing spy usage and complexities which I found very useful. It begins with a description of spies and then goes into examples of usage.
The first example is used on a real object – a list – which a method is then called upon twice, and then Mockito’s verify is called twice with the same method. This allows for “spying” upon an object, hence the name spies. There is then some detail about the @Spy annotation, stubbing a spy, and then the actual differences between a Mock and a Spy. Chiefly, this difference is that a mock uses the Class of a Type, like a “base-bones shell instance,” while a spy “will wrap an existing instance” and can track it in various ways. This is also followed by a blurb about the NotAMockException from Mockito, which wasn’t immediately necessary for my purposes I don’t think.
Overall, I thought this was a great, useful article. It prepped me for continued learning within Mockito and I felt comfortable understanding it. I think I have a good basic understanding of spies and would be ready to begin using them in testing. They’re a good way to analyze object/method behavior and have useful applications for when a project is only partially written. I look forward to using spies in the future. Most of what I’ve learned within the world of Mockito has been super interesting thus far.
I wanted to learn more about mock testing after working with it and other testing approaches in the recent activities. There are some parts of mock testing that I am think I could understand better after doing some more reading on it.
The process and purpose of mocks make sense to me, where mocks are used to simulate behaviors for tests and focus more on the code itself. Because of this, mock testing is based on the behavior of code while unit testing is state based. State based is for the values you get from the code while behavior based is for checking if the right method is called. Behavior based is important because you can get the right values, but you might not call the right method if there are methods with similar return types and parameters.
I think where I had some misunderstandings at first was when it came to dependencies. Mock testing is based around dependencies and it seem like mock testing will be more difficult if you have some trouble with dependencies. I haven’t worked as much with dependencies as other areas with code, and I had a little trouble with it in a past assignment.
Dependencies are other programs that the main code relies in order to work, with JUnit being one of the main examples that we work with. Where I have some trouble with this is less with understanding the concept and more with some parts of how it works in the IDEs. I sometimes get errors from not properly setting the dependencies and get stuck on them for a while. Mockito is another one we have started working with that was giving me some trouble in a similar way.
I have a better grasp on dependencies after reading more about them and mock testing, and I hope to overcome problems that I have working with mock testing as we continue in the activities.
After a piece of software, whether that be the project as a whole or simply one component as part of a system has been completed, it makes sense to give the final product a sort of “look-over” to make sure that everything is working as intended and meets any required specifications. This is something which generally seems to be good practice to follow anyway in my opinion, since it could be easy to overlook something during the development process. It can be easy to become hyper-focused on one thing or another when working on larger projects with multiple components, which could lead to the potential to forget or leave-out some minor (or major) requirements, introduce bugs or flaws, or otherwise implement things in a way which is inefficient or ineffective.
This is where the more formally defined process of “Code Review” is beneficial. While looking into the process, I found this helpful writeup from Atlassian: (https://www.atlassian.com/agile/software-development/code-reviews) which goes into great detail regarding the practice. Overall, the process involves looking for any clearly visible bugs or issues with the code, as well as considering logic errors and concerns. The finished code/components is compared to the requirements of the project, and any test-cases or testing methodologies used are evaluated (how complete is the test coverage, should any new tests be added?). Additionally the code is compared to preexisting code (referred to as “style guidelines” in the post by Atlassian) which exists within the same space or project to promote consistency and cohesiveness.
Compared to the process I described earlier, the formal definition of Code Review provided by Atlassian hits many of the same key areas. The finished piece or product is examined in relation to any requirements, testing/test cases are examined to determine coverage and whether any additional tests are necessary, and bugs, errors, or logical flaws are sought out and fixed where they have occurred. I would say that the last point regarding “style guidelines” is less important, especially in solo-development environments where you might not be working in a team. Regardless it still seems like a good thing to keep in mind when working in larger group applications. Code review is a helpful and largely effective way to take a second look after the main development process is complete and make sure that the final product is what it was intended to be.
Mr. Bugayenko talks about the structure of a simple Java project and return values of various methods. In the article, it shows how much work it would take to mock a program that has multiple levels of abstraction and if we did mock everything, it would obscure the meaning of our tests because our tests will would have multiple levels of abstraction and makes it that much harder to see what exactly we are testing. In the end, it would also make the test file much longer than our main file. In situations like this, it would be easier to use fakes.
According to the article linked above, integration testing is a type of software testing that focuses on how well different components of a system interact with each other. For example, when assembling a pen, one would ideally test if all the parts for a pen fit together like the cap, ink cartridge, tail, among other parts. There are two main types of integration testing called unit integrated testing and system integrated testing respectively. Unit integrated testing focuses on testing the interactions and interfaces between integrated components. System integrated testing focuses on testing the interactions and interfaces between systems. There are also four ways to approach integration testing called Big Bang, Top Down, Bottom Up, and Sandwich or Hybrid. Big Bang is when most if not all components are tested at once such as all the functionalities in a system. Top Down is when top-level components are tested first followed by lower-level components. Bottom Up is the opposite where low-level components are tested first followed by higher-level ones. Sandwich, also called Hybrid, is combination of both Top Down and Bottom Up approaches.
For my software capstone, me and three other team members are working on an inventory system. Each of us are working on different components; for example, I am working on the backend API and a messaging system. Right now, I’m in the middle of testing the API by itself, but I’ll have to test it in conjunction with other components like the messaging system, the rest of the backend, the frontend, and also the other systems that are being developed simultaneously. One could call this a form of integration testing, focusing both on the unit and system levels. I feel I’ll take a Hybrid approach since what’ll be tested first is whatever I finish first and what my teammates finish first.
Christian Shadis, CS-443 Self-Directed Blog Post #4
For students like me with limited experience writing JUnit tests, it becomes easy to get lost in the repetition of “Setup, Exercise, Verify” and not realize that a simple, understandable feature of JUnit testing, annotations, can be used to improve the functionality and efficiency of your JUnit Test Suites. In this post I will explain the uses five JUnit 5 annotations and demonstrate them. The following annotations will be explained:
We begin with an extremely simple Java class “Book” and its similarly simple counterpart “BookTest”. See the code for the two classes below.
public class Book {
private int pages;
private String author;
private int pubYear;
public Book(int pages, String author, int pubYear){
this.pages = pages;
this.author = author;
this.pubYear = pubYear;
}
public int getPages(){ return this.pages; }
public String getAuthor(){ return this.author; }
public int getPubYear(){ return this.pubYear; }
public void setPages(int pages){ this.pages = pages; }
public void setAuthor(String author){ this.author = author; }
public void setPubYear(int pubYear){ this.pubYear = pubYear; }
}
import static org.junit.jupiter.api.Assertions.*;
import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.api.Test;
public class BookTest {
@Test
void testConstructor(){
Book book1 = new Book(100, "John Doe", 1999);
boolean result = book1 instanceof Book;
assertTrue(result);
}
@Test
void testGetPages(){
Book book1 = new Book(100, "John Doe", 1999);
int result = book1.getPages();
assertEquals(result, 100);
}
@Test
void testGetAuthor(){
Book book1 = new Book(100, "John Doe", 1999);
String result = book1.getAuthor();
assertEquals(result, "John Doe");
}
@Test
void testGetPubYear(){
Book book1 = new Book(100, "John Doe", 1999);
int result = book1.getPubYear();
assertEquals(result, 1999);
}
@Test
void testSetPages(){
Book book1 = new Book(100, "John Doe", 1999);
book1.setPages(150);
assertEquals(book1.getPages(), 150);
}
@Test
void testSetAuthor(){
Book book1 = new Book(100, "John Doe", 1999);
book1.setAuthor("Jane Smith");
assertEquals(book1.getAuthor(), "Jane Smith");
}
@Test
void testSetPubYear(){
Book book1 = new Book(100, "John Doe", 1999);
book1.setPubYear(2001);
assertEquals(book1.getPubYear(), 2001);
}
}
@BeforeEach
The first annotation we will look at is @BeforeEach. This annotation is used for repetitive code that is executed during each test case. If you examined BookTest closely, you might have noticed that the first line of every test case is identical. This makes our suite the perfect candidate to use @BeforeEach. In order to implement, we must create a setUp test method, and put book1 as an instance variable – this way, scope does not become a nuisance. See the revised BookTest.java file below.
import static org.junit.jupiter.api.Assertions.*;
import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.api.Test;
public class BookTest {
Book book1;
@BeforeEach
void setUp(){
book1 = new Book(100, "John Doe", 1999);
}
@Test
void testConstructor(){
boolean result = book1 instanceof Book;
assertTrue(result);
}
@Test
void testGetPages(){
int result = book1.getPages();
assertEquals(result, 100);
}
@Test
void testGetAuthor(){
String result = book1.getAuthor();
assertEquals(result, "John Doe");
}
@Test
void testGetPubYear(){
int result = book1.getPubYear();
assertEquals(result, 1999);
}
@Test
void testSetPages(){
book1.setPages(150);
assertEquals(book1.getPages(), 150);
}
@Test
void testSetAuthor(){
book1.setAuthor("Jane Smith");
assertEquals(book1.getAuthor(), "Jane Smith");
}
@Test
void testSetPubYear(){
book1.setPubYear(2001);
assertEquals(book1.getPubYear(), 2001);
}
}
@AfterEach
Analogous to @BeforeEach, @AfterEach will execute following each test case. There is no obvious implementation in our code, but I will demonstrate its use nonetheless. We can use @AfterEach to update a count variable indicating how many tests are run. We add a print statement to our setup method to print the test number, and add count as an instance variable, initialized to 0. @AfterEach will allow us to increment the count after each test case is run. Create a method tearDown() with the @AfterEach annotation, and add “count++;” as the body. The count declaration must also be revised to a public static modifier. The new instance variables, setUp(), and tearDown() methods are shown below.
public class BookTest {
Book book1;
private static int count = 0;
@BeforeEach
void setUp(){
book1 = new Book(100, "John Doe", 1999);
System.out.println("Running test " + count);
}
@AfterEach
void tearDown(){
count++;
}
.
.
.
}
We now have methods that execute before and after every single test case.
@BeforeAll
@BeforeAll is used as an initialization method that executes before any other method. It is the first method in BookTest that will be run. We can use it in this example to instantiate our count variable to 0, and leave just the variable declaration with the instance variables. See the modified instance variables and new init() method below.
public class BookTest {
private static int count;
Book book1;
@BeforeAll //runs first
public static void init(){
System.out.println("BeforeAll running: count set to 0.");
count = 0;
}
.
.
.
}
@ParameterizedTest
This annotation is useful for running test cases with multiple inputs. For example, imagine the developer of BookTest is unsure whether testSetAuthor() will pass when multi-word strings are passed as arguments. This scenario is the perfect situation to use a parameterized test. When using @ParameterizedTest, however, one must be sure to specify the source of the arguments, which is accomplished using the annotation @ValueSource. See the example below.
This annotation is used for when tests need to be run multiple times. For example, if we wanted to run testGetAuthor three times, we would use this annotation. When using @RepeatingTest, we must pass a parameter value which determines the number of repetitions, and name which names each repetition. See the example below.
@RepeatedTest(value = 3, name = "Repetition {currentRepetition}")
void testGetAuthor(){
String result = book1.getAuthor();
assertEquals(result, "John Doe");
}
Summary
We have now taken an extremely simple JUnit 5 test suite and used five different annotations to make the suite a bit more complex and comprehensive. There are many more annotations, all of which can be found in the JUnit documentation here. We can now run tests multiple times, create setup and teardown methods, and parameterize tests to run them with multiple argument inputs.
public class Book {
private int pages;
private String author;
private int pubYear;
public Book(int pages, String author, int pubYear){
this.pages = pages;
this.author = author;
this.pubYear = pubYear;
}
public int getPages(){ return this.pages; }
public String getAuthor(){ return this.author; }
public int getPubYear(){ return this.pubYear; }
public void setPages(int pages){ this.pages = pages; }
public void setAuthor(String author){ this.author = author; }
public void setPubYear(int pubYear){ this.pubYear = pubYear; }
}
import static org.junit.jupiter.api.Assertions.*;
import org.junit.jupiter.api.*;
import org.junit.jupiter.params.ParameterizedTest;
import org.junit.jupiter.params.provider.ValueSource;
public class BookTest {
private static int count;
Book book1;
@BeforeAll
public static void init(){
System.out.println("BeforeAll running: count set to 0.");
count = 0;
}
@BeforeEach
void setUp(){
book1 = new Book(100, "John Doe", 1999);
System.out.println("Running test " + count);
}
@AfterEach
void tearDown(){
count = count + 1;
}
@Test
void testConstructor(){
boolean result = book1 instanceof Book;
assertTrue(result);
}
@Test
void testGetPages(){
int result = book1.getPages();
assertEquals(result, 100);
}
@RepeatedTest(value = 3, name = "Repetition {currentRepetition}")
void testGetAuthor(){
String result = book1.getAuthor();
assertEquals(result, "John Doe");
}
@Test
void testGetPubYear(){
int result = book1.getPubYear();
assertEquals(result, 1999);
}
@Test
void testSetPages(){
book1.setPages(150);
assertEquals(book1.getPages(), 150);
}
@ParameterizedTest
@ValueSource(strings = {"Jane", "Jane Smith"})
void testSetAuthor(String str){
book1.setAuthor(str);
assertEquals(book1.getAuthor(), str);
}
@Test
void testSetPubYear(){
book1.setPubYear(2001);
assertEquals(book1.getPubYear(), 2001);
}
}
Although I may not have stated it explicitly, I wrote my previous post regarding using boundary values during testing on programs that are, more or less, complete with all the necessary functions having been implemented. That can be the case for a programmer who is working on software that is relatively small and can be coded to completion by a single individual. Though, one would ask, what happens when the project’s scale is large enough that multiple people would need to be assigned to code certain chunks of code that would be combined into a complete piece of software? The programmers may have some guidelines on or a semblance of how the software should work as a whole, including some of the parts they aren’t actually assigned to implement, yet they may not have the time or resources to work on the other functionalities on top of the ones assigned to them. Yet, to ensure that their assigned functionalities work according to specifications, they will still need the missing code to be implemented for which they may not know how to implement, mostly because of certain specifications that are expected to be met yet are unknown to the programmer. And the cycle continues.
In such cases, one technique that programmers can utilize in order to test their Work-In-Progress code without having to manually implement the missing code is test doubles. Essentially, programmers use test doubles as a way to make up for the missing functionalities by implementing simplified objects or methods such that the can still be run with minimal amount of code that can be easily replaced by the actual final product. Test doubles include, but are not limited to, the following: 1) Dummies 2) Stubs 3) Fakes 4) Mocks
I will be discussing Mocks along with Mockito in more detail in a later post. Instead, in this post I will be focusing on the first three types of test doubles that are given above. What I will mention about mocks, however, is that they can utilize the Mockito testing framework to automate and simplify the process of testing to a more significant degree compared to the previous three types. It goes without saying, however, that each approach has its own merits depending on the needs.
Dummies or dummy objects are objects or procedures with minimal, if any, implementation, making it by far the simplest form of test double implementation. In other words, dummy objects can be considered a bare-bone implementation of code that is sufficient enough to help in testing. Below is a simple example of a dummy object:
public DummyObject(){ } Essentially, the above dummy object is more than enough code even though lacks the implementation. Dummy objects are created to be used when other methods may require parameters, though in this particular case we are not actually creating the objects ourselves.
Stubs are (or I at least believe they are) a more common method of testing methods. Essentially, stubs contain slightly more (yet still bare-bones) implementation than dummies, which is more often than not a predefined return value that corresponds to the return type. Below is an example of a stub method: public double average(double a, double b){ return 3.1; } In the above example, the method will not do any calculations. Instead, it will return an arbitrary value that is relevant to the return type, without any other significance whatsoever. Stubs are used to test if the method actually responds when being called during testing, rather than if the method produces an expected result after running a specific set of computations. I have found myself using stubs when programming in C earlier in my studies just to make sure if other parts of my code can be compiled properly, though now I know that stubs may also be used in unit testing as well.
The last type of test doubles I will be talking about is fakes. Fakes or fake objects have some implementation, though this implementation is often different from that in the end product. Fake objects do not actually use resources (such as data from a database) but may use some arbitrary data in order for the object to be used in testing. An example of a fake object is given below:
Compared to stubs and dummies, fakes contain more implementation, though they are much “lighter” to use than actual objects that contain data that may need to be retrieved in one way or another.
Overall, which of the above implementations may need to be used in testing depends solely on the programmer’s judgement and the state of their code. When combined with other testing techniques (such as boundary value testing), unit testing may provide enough insight to the developers in order to make the appropriate adjustments to their software, be it in regards to fixing faulty code or implementing better optimization. Like I mentioned earlier, each technique has its own merits depending on the circumstances. Moreover, in my next post I will be discussing Mock objects, the Mockito testing framework, and some of the advantages Mockito brings to software testing.
