What Do I Test? – Weak, Robust And Edge Case Testing

Deciding on what to test can be a little tricky. If you only test a program using inputs that are “valid” you could find a bug later due to the fact an “invalid” input was entered or used. These invalid and valid value tests focus on the very edges of a program, earning itself the term edge case testing.

On the other hand you may have certain manipulations or branches in a program that will need to be tested. Those tests can also have a sort of interior version of edge case testing.

To give a more concrete example later i will use an example of having a make believe chance of rain calculator. We will keep it very basic just to understand the idea.

The following will be the specifications of the program.
temp < 60 = 0%
temp >= 60 = 50%
humidity < 50 = 10%
humidity >= 50 = 30%
chance of rain = temp% + humidity%

Below is a picture from a class activity that depicts this topic. There are 4 parts that are important to understand before you start deciding on and writing test cases.

Weak Normal:
A weak normal test will focus on valid inputs, checking the basic calculations or manipulations of a function. It will not test for all of the possible combinations.

Strong Normal:
A strong normal test will focus on all of the possible different outcomes. One would view it as a more in depth version of a weak testing. The amount of tests will very quickly add up. However it becomes more obvious how important it is to test every possible outcome as a program grows and has a higher chance to produce bugs.

Weak Robust:
A weak robust is more comparable to a weak normal. Reason being that it wont test for every possible outcome like a strong normal would. However the difference between a weak robust and weak normal is that the robust will incorporate some tests using invalid inputs.

Using our example of the rain chance calculator. Would you want to make sure that the chance of rain is only 30% if the humidity has a rating of 3,000? (usually 100% humidity is raining)

Strong Robust:
Last but absolutely not least, we have the strong robust test cases. This version of testing is where every single possible input and output is accounted for and testing. This is truly the strongest version of testing and can get extensive pretty quickly. Before diving into the tests themselves using the made up rain chance calculator. It is important to take a look at the formulas below

If we use the specs for our rain calculator and the formulas above, possible Strong robust test cases might look something like:

checking temp at -1, 0 and 1
checking temp at 59,60,61
(technically temperature wouldn’t have a maximum or minimum value to test)

checking humidity at -1, 0, 1
checking humidity at 49, 50, 51
checking humidity at 99, 100, 101

With all of those inputs accounted for you would then make sure the corresponding outputs for the final calculation are correct.

In conclusion it is obvious that strong robust testing is the superior option. However with a project of smaller scope, using a weak or normal version of testing could also be sufficient.

From the blog Mikes CS Journey by Michael St. Germain and used with permission of the author. All other rights reserved by the author.

Why Should I Test?

The thought process behind why to test was, in my opinion, an interesting and fun topic to explore. This will focus more on the abstract process of basic testing it should be obvious as to why you would want to make sure a program functions properly. Starting with a photo of a diagram we used in class.

This first diagram depicts how my brain used to think of programming. It can’t be understated just how much has been learned and how my thought process has changed due to learning how to test my programs. The diagram has 3 simple regions. 1 has specifications, 2 has implemented code and 3 would be the completed project, The implemented code meets specification. Short, sweet and simple.

This second diagram fully represents how i think now, and how everyone should be viewing it from the start in my opinion.

In the diagram there are three main circles. The top left returns and represents the specification of a program. That is what the end goals of what a project should be able to do. The top right also makes a comeback and is what has been implemented so far in a theoretical program. The bottom circle indicates test cases.

The team i worked with to answer questions regarding the diagram had a lot of fun poking at the over lapping aspects with our initial viewing of them.

Area 1 would be a completed project. The specifications have been met and implemented and there are tests that met expectations and pass (verified).

Area 2 has specifications and some of it is implemented, but there are no tests or the tests have not passed.

Area 3 is where we started it started to become a little silly. There is Implemented code that has tests but no one asked for it.

Area 4 I remember actually laughing at, would have some specifications and tests. However it would have no code to use those tests on.

The other areas follow a similar pattern ending with 7. Which we referred to as just a “useless” testcase.

As expected, the end goal of someone working on a project would be to end up in circle one.

Now that I’ve completed the course and reflect upon this diagram and our responses to what each area represents. It is a good measure of our inexperience with test driven development. In area 4 specifically, having some set specifications and tests for them made me laugh because what would be the need to have those tests. It wasn’t until later that i made the connection. Now i understand that would be a great point to start writing code from. Area 2 is where i would of thought to begin at the time.

As for area 7 and the “useless” test cases. I no longer feel there is such a thing, after the experience shared in my blog post Test Driven Development (part 2).

From the blog Mikes CS Journey by Michael St. Germain and used with permission of the author. All other rights reserved by the author.

Test Driven Development – Why it Might Feel Backwards And How to Fix that (part 2)

At the end of the semester my group faced a true hands on experience of why TDD is so popular. While working on a group project to explore testing in languages other then JUnit, i took the initiative to try and create a basic java script program. I had spent quite a good amount of time writing and rewriting things until i thought we had a good program to write tests for together.

It didn’t take long for us to realize the error in our thinking had struck again. the code i had worked to create couldn’t even be tested. Luckily another group member was able to guide us into how the tests should be written.

Thus began the rewriting of the code i thought i had already written. Long story short the main issue was i had gone too far into playing with the HTML file. and we couldn’t figure out how to mock an HTML file (a learning experience for another time).

What we did was write out all of our tests first using mocha and chai. We quickly learned that the display for these tests was going to make the process quite a bit less painful. We started pulling functions and other pieces from the original code and put them into a separate file while running our tests on them.

This was when we realized what we had done was literally TDD, and if i had done it in the correct order it would have been much simpler and less time consuming.

good sources for mocha and chai with examples:
https://www.browserstack.com/guide/unit-testing-for-nodejs-using-mocha-and-chai

https://www.chaijs.com/

https://mochajs.org/

From the blog Mikes CS Journey by Michael St. Germain and used with permission of the author. All other rights reserved by the author.

Test Driven Development – Why it Might Feel Backwards And How to Fix that (part 1)

When first being introduced to TDD (Test driven development) I was in class with my group. we all had the same feeling, this feels like were doing everything backwards!

We had been used to writing semi-clean code and then testing for it afterwards. Which in our inexperienced minds made perfectly good sense. Write some code. Then test that code.

However, after doing an assignment for class we started to realize the power of TDD. We were able to better organize our thoughts and methods before writing the code. A couple of us agreed that it kind of felt like having pseudo code and being able to write TESTABLE code was a skill. Lastly, when it came to refactoring things. We never had to rewrite entire tests, and we already had tests to work with to ensure our changes were working properly.

Some helpful resources:
https://stackoverflow.com/questions/14891967/understanding-stubs-fakes-and-mocks
(explains some of the terms needed)

https://junit.org/junit4/
(examples to follow for JUnit)

From the blog Mikes CS Journey by Michael St. Germain and used with permission of the author. All other rights reserved by the author.

J Unit Stubs

*Through my teams experience we were getting a little confused on the differences of mocks and stubs, so i would like to explain each one separately.

Simply put, you can view a stub as a type of mock with more control and is a little easier then making a mock of an object. This picture helped me realize the difference.

As can be seen, by using a stub you can pretty much do everything a mock can, without having an entire mocked object. generally a stub will have the minimal amount of parts needed in order to make tests pass. the tests themselves are focused on the class itself, rather then the manipulation of an object. A stub will usually return a predetermined value. Hence why it can be viewed as a type of mock, but is still its own thing.

Sources:
https://stackoverflow.com/questions/31890991/how-to-use-stubs-in-junit-and-java
(photo source)

https://semaphoreci.com/community/tutorials/stubbing-and-mocking-with-mockito-2-and-junit
(helpful for understanding)

From the blog Mikes CS Journey by Michael St. Germain and used with permission of the author. All other rights reserved by the author.

J Unit Mocks

*Through my teams experience we were getting a little confused on the differences of mocks and stubs, so i would like to explain each one separately.

In normal words, a mock implements some behaviors that may not already exist in the code you’re trying to test. I’ve found a picture to help illustrate.

It is creating a fake object that will use code you are testing. The tests make sure that object is interacting with the “class under test”, correctly. The mock will not have preprogrammed return values and are used to find side effects and whether the correct methods are being called.

To further explain, one may find themselves working on a portion of code where they don’t have an object that is meant to be used prepared already. These objects could also be a giant database, requiring you to access a large amount of data over and over while testing a much smaller class. Using a mock in this case would speed up the process. Focusing only on the class you want to test for behaviors.

Sources:
https://stackoverflow.com/questions/31890991/how-to-use-stubs-in-junit-and-java
(photo source)

https://semaphoreci.com/community/tutorials/stubbing-and-mocking-with-mockito-2-and-junit
(helpful for understanding)

From the blog Mikes CS Journey by Michael St. Germain and used with permission of the author. All other rights reserved by the author.

Week 18B – C Testing

For this week, I wanted to look at how different languages handle test cases, and I’ll continue with one I’m not the most familiar with, C! I’ve worked in small amount of C in classes at Worcester State, but have little experience outside of that. I feel like this is a good topic to discuss as knowing how other programming languages handle unit testing would be a great way to expand my knowledge when it comes to furthering my understanding of it within Java.

If you haven’t already read my other blog post on Python testing, feel free to read it right here!

Week 18A – Python Testing

For learning about unit testing in C, I consulted this article on the subject: https://interrupt.memfault.com/blog/unit-testing-basics

It seems like unit testing in C is a lot more barebones compared to Java, which in my experience utilizing C, makes sense for the language. A lot of features primarily used in Java, like object-oriented structures aren’t available in C (to my understanding, could totally be wrong).

For one major aspect, there seems to be only one assertion command in C, just simply “assert”. Theres no assertTrue, assertFalse, assertThrows, or assertEquals, just simply “assert”. And from the example given below: 

#include <assert.h>

// In my_sum.c
int my_sum(int a, int b) {
  return a + b;
}

// In test_my_sum.c
int main(int argc, char *argv[]) {
  assert(2 == my_sum(1, 1));
  assert(-2 == my_sum(-1, -1));
  assert(0 == my_sum(0, 0));
  // ...
  return(0);
}

It seems the “assert” function comes from the <assert.h> library, much like the JUnit librarys used in Java. But more importantly, it seems that “assert” is the equivalent of “assertEquals”.

It also seems like Unit Testing in C is best implemented with tools outside of a compiler for C. The ones mentioned in the article in specific were CppUTest, Unity, and Google Test. For the rest of the article, the use examples using CppUTest. It was interesting to hear one of the options being called Unity, which is the name of a game engine, which, while not written in C, is written in a mixture of C# and C++, which are both offshoots of C. Makes me wonder how testing in a gaming engine works, perhaps it’s something to look at in a future blog post, hint hint, wink wink.

CppUTest seems to implement the same SetUp() and Teardown() functions that JUnit can employ, which is really good, as these methods are important for testing multiple methods. It also seems to have more then just an Equals assertion, even though the example used is another equals example.

This gets me more interested in C, as I have been told understanding C allows you to understand other languages much more clearly. Perhaps I’ll take a deeper dive some day, who knows! Until next time, my readers~!

From the blog CS@Worcester – You&#039;re Telling Me A Shrimp Wrote This Code?! by tempurashrimple and used with permission of the author. All other rights reserved by the author.

Week 18A – Python Testing

For this week, I wanted to look at how different languages handle test cases, and I’ll begin with the one I’m the most familiar with, Python! I’ve worked with Python in small amounts in the past, and have an understanding a lot of it’s syntaxes are similar to java’s, albeit simpler. I feel like this is a good topic to discuss as knowing how other programming languages handle unit testing would be a great way to expand my knowledge when it comes to furthering my understanding of it within Java.

For this, I’ll be looking at the official page for unittest on Python’s website, here:

https://docs.python.org/3/library/unittest.html

Right off the bat, I’m really interested in the fact that unittest is actually based directly off of JUnit! Which means a lot of the syntax, formatting, and framework is quite similar, just modified to fit the mold of Python.

Looking at the snippet they gave as an example…

import unittest

class TestStringMethods(unittest.TestCase):

    def test_upper(self):
        self.assertEqual('foo'.upper(), 'FOO')

    def test_isupper(self):
        self.assertTrue('FOO'.isupper())
        self.assertFalse('Foo'.isupper())

    def test_split(self):
        s = 'hello world'
        self.assertEqual(s.split(), ['hello', 'world'])
        # check that s.split fails when the separator is not a string
        with self.assertRaises(TypeError):
            s.split(2)

if __name__ == '__main__':
    unittest.main()

In this, it seems the way you define test blocks is by having a class with (unittest.testcase) and then doing “def” to define each test case.

Even the assertions are the same and written near identically, as the first three use assertEqual, which is identical to javas assertEquals, minus the s, and assertTrue and assertFalse, which are also identical to their java counterparts. assertRaises, which is used in the third test, seems to be Python’s equivalent to assertThrows, however, it seems to be a bit different in comparison. assertRaises seems to identify a specific kind of exception being raised, whereas assertThrows would just identify any exception in general.

The last line also is a block of code that allows an easy way to run all the tests, so when you run unittest.main() in a command line, it will automatically run all the tests and display the results.

There also seems to be a whole bunch of different command line options to display results and modify the ways in which its run. As an example, theres “-v”, which stands for verbosity, much like the bash command, which shows the results of each individual test being run, like below:

test_isupper (__main__.TestStringMethods.test_isupper) ... ok
test_split (__main__.TestStringMethods.test_split) ... ok
test_upper (__main__.TestStringMethods.test_upper) ... ok

----------------------------------------------------------------------
Ran 3 tests in 0.001s

OK

It seems extremely interesting and makes me want to learn more Python, which would definitely help me in my career in all sorts of ways! Next blog we will be looking at how unit testing works in C. Until then!

From the blog CS@Worcester – You&#039;re Telling Me A Shrimp Wrote This Code?! by tempurashrimple and used with permission of the author. All other rights reserved by the author.

Behavior Driven Development

Behavior Driven Development ( BDD ) is a test practice that makes sure there is good quality by automating test before or during system behavior specification. BDD test focuses on facing scenarios that describe the behavior of a story, feature, or capability from a user’s perspective. When the tests are automated they make sure that the system constantly meets the required behavior.

The Behavior Driven Development Process

The BDD process has three phases to it. The discovery phase, formulation phase, and the automation phase.

1.) Discover phase: This phase is where the user creates the initial acceptance agenda for the feature. This phase is usually done in a collaborative manor, each team member is contributing.

2.) Formulation phase: This phase is where the acceptance agenda sets into detailed acceptance tests, as the backlog item gets closer to implementation. This phase also incorporates specific examples of the behavior.

3.) Automation phase: This phase is where automation tests are automated to run constantly. This is to make sure that the new system supports the new behavior.

Benefits of Behavior Driven Development

1.) Early detection of errors / defects: When you automate tests in the early stages of development process, you can identify and address the issues. BDD allows for the early detection of defects.

2.) Faster Flow and Time: when using BDD, you can reduce the errors, rework, and replan. BDD accelerates the flow of the development process. Developers can produce features / products faster and more efficiently.

3.) Stronger Test Coverage: BDD allows for a more comprehensive test coverage that focuses on the user behavior and scenarios. Both common and edge cases are tested as well.

4.) Clear understanding: BDD can be plain and clear to understand, because specific scenarios are used to describe the behavior from a user’s point of view. This helps the development to fully understand the requirements and whats going on.

Why I chose this resource

I chose this article ” Behavior Driven Development” because it provided a detail look of a very important test method that goes in conjunction with the technical and business aspect of testing. Understanding BDD is important in today’s society of software development, for giving an efficient and more user friendly user products.

Personal Reflection

This article increased my understanding of BDD and the use of it in software development. I learned a lot about how BDD strengthens collaboration and communication between the business side of things and the technical side of things. This helps to ensure that user’s expectations and requirements are met. The new found knowledge will be extremely valuable in my future endeavors because I will incorporate this method in my future projects. This will help to improve the development process and product efficiency and quality. Also, by using BDD I can make sure that all requirements and specifications are met.

The full article is here: https://scaledagileframework.com/behavior-driven-development/

From the blog CS@Worcester – In&#039;s and Out&#039;s of Software Testing by Jaylon Brodie and used with permission of the author. All other rights reserved by the author.

Static Testing vs. Dynamic Testing

Testing in software development is important because it helps to deliver efficient and user friendly products to the end user. It also provides the developers with a chance to improve upon the product. Static and Dynamic testing are two important techniques used in software development.

Static Testing

Static Testing has various names like Verification Testing, Non-execution Testing, etc. This testing technique is used to identify defects in software without actually executing the code. This method usually includes manual and automated evaluation of the software and the code. Developers use this method usually in the beginning stages of the development process to catch issues early on, which will also lead to be easier and cheap to fix. This method focuses on reviewing the test cases, test scripts, test plans, and source code.

Static Testing Techniques

1.) Informal Reviews: Developers review each of the documents and give feedback

2.) Walkthroughs: Someone presents the product to the team and someone else takes notes.

3.) Technical Reviews / Code Reviews: review the technical specifications and the source code to make sure everything meets the requirements and standards.

4.) Inspection: Check for defects. Developers usually review the process with a checklist to help identify and record for defects.

Dynamic Testing

Dynamic Testing is a technique that analyzes the dynamic behavior of the code by actually executing it. This method makes sure to check that the software functions correctly and that there are no underlying issues / conditions. Sometimes developers use this method in conjunction with black box or white box testing to provide more realistic results.

Dynamic Testing Techniques

1.) White Box Testing: Examines the internal code structure. You need to actually have the internal code (source code)

2.) Black Box Testing: Checks the functionality without the actual internal code (source code) .

Benefits of both Static Testing and Dynamic Testing

1.) Early detection of defects

2.) Cost efficient

3.) Showcases runtime errors

4.) Reliability

Why I picked this Resource

I chose the article “Static Testing vs. Dynamic Testing” because this article gave me a more detailed and in depth look between two very important testing methods that are currently being used in todays society. It is very important to understand these two testing methods in the software development process because they can deliver efficient and user friendly products to the end user. This article also aligns with what we have learned in the course, making it relevant to talk about and to understand.

Personal Reflection

This article deepened my understanding of static and dynamic testing. I was able to learn a lot about these two testing methods that I did not know, even the many benefits that each method has. Knowing how crucial these two methods are in the software development process and what I know now, this knowledge will help me on my future endeavors when approaching new projects in regards to testing .

The full article is here: https://www.geeksforgeeks.org/difference-between-static-and-dynamic-testing/

From the blog CS@Worcester – In&#039;s and Out&#039;s of Software Testing by Jaylon Brodie and used with permission of the author. All other rights reserved by the author.