Category Archives: CS-443

New Tricentis qTest Case Studies Highlight Testing’s Critical Role in Agile Transformation

Hello again everyone. For my second blog of the semester (Technically third because of intro post) I am using another article by Lanier Norville. Last week, I wrote about her article on testers becoming agents of change. This week, however, I am going to be writing about some Tricentis qTest Case Studies. I picked this article because it talks about Agile Transformation, and I am personally fond of agile frameworks.

Once again Norville leads off with a nice and small yet appropriate introduction on what she is discussing. In this case, she is talking about how companies are transforming agile and DevOps. She uses her case studies to show the “critical role” (Norville) of testers.

The first case study involves a payment processing technology provider. The VP of Test Engineering, Nick Jones attended an event on DevOps and he decided that his organization needed to transform as well. Norville then discusses how payment options have an effect on whether customers end up buying something or not. Jones developed a DevOps roadmap with his team and according to Norville they have reduced the time of delivery from “14 hours to 4 minute” (Norville) This is interesting because it is a huge drop in time which is huge for a company that is constantly making deliveries.

The next study isn’t as much of a success story as the last study, however it still seems like it is helpful. The University of the West of England switched to Tricentis qTest before using an agile framework. The head of testing, Heather Daniels that she “needed to implement a test management tool” (Norville) She needed to do this in order to maintain everything that the school used. (Library systems, eLearning systems, etc.) As of now, according to Norville, two of the organizations have switched to agile. It is a big change according to Daniels, and the University still isn’t used to making small functional deliveries, but it seems like they are getting the hang of it. I like that they switched because now they are “prioritizing the things that are most important to users first.” (Norville)

This article, in my opinion, was a little bit more difficult for me to follow, but it was still a very well written article overall. I did like how well she described each of the scenarios and what was done with qTest to transform into an agile framework. It is obvious that the article is supporting the websites own product, but that is what companies are supposed to do. Overall, this was a good read, but for my next blog, I will probably be visiting another website.

New Tricentis qTest Case Studies Highlight Testing’s Critical Role in Agile Transformation

From the blog CS@Worcester – My Life in Comp Sci by Tyler Rego and used with permission of the author. All other rights reserved by the author.

Decision table test design technique

Decision table also known as cause effect
table it is one of the black box test design techniques

what are that black box design techniques?

Black Box Testing is a software testing method in
which the internal structure/ design/ implementation of the item being tested
is NOT known to the tester. The black box is designed techniques are widely used as a best
practice in the industry. Black box techniques are used to pick the test cases
in a systematic manner by using these techniques we could save lots of testing
time and get the good coverage.

 Decision Table Test Case Design Technique:

This test technique is appropriate for functionalities where it has the logical relationships between the inputs (if-else logic). In Decision table technique, we are working with a combination of inputs. To identify the test cases with decision table, we have conditions and actions. We take conditions as inputs and actions as outputs.

As an example, for that lets practice on a small test case which is Login Screen

Let’s identify the conditions our inputs:

The user will type in the correct username and password, the user will be logged in successfully

If any of the input is wrong, an
error message will be displayed.

Now let’s divide our condition in to
true, false, error, and home screen

  • T for true – Correct username/password
  • F for false– Wrong username/password
  • E for error– Error message is displayed
  • H for home– Home screen is displayed

That leads us to the cases we are going to create:

  • Case 1 – Username and password both were wrong. The user is shown
    an error message.
  • Case 2 – Username was correct, but the password was wrong. The user
    is shown an error message.
  • Case 3 – Username was wrong, but the password was correct. The user
    is shown an error message.
  • Case 4 – Username and password both were correct, and the user
    navigated to homepage

Now let’s try to put all of this
into one decision table  

conditions Rule1 Rule2 Rule3 Rule4
Username F T F T
password F F T T
Output E E E H

And this is the way to create a decision table

Advantage of decision table technique:

  1. every complicated business can be converted easily into a
    test scenarios & test cases using this technique.
  2. The first iteration is used as input table for next
  3. Once you get the idea of how the test works it become simple
    to understand and anyone can use this method.
  4. It gives a full coverage of test cases which help to
    reduce the rework.



Works Cited

From the blog CS@Worcester – Shams's Bits and Bytes by Shsms Al Farees and used with permission of the author. All other rights reserved by the author.

Testing Doubles

When testing a system, known as the System Under Test or SUT,
the system needs to have access to its dependencies but often the access given
is just clever mimicry. Testing doubles are used to replace a system’s dependencies
with varied functionality. The article “Test Doubles — Fakes, Mocks and Stubs,”
by Michal Lipski, explains three out of five of the types of testing doubles defined
by Gerard Meszaros, fakes, stubs, and mocks. The author provides diagrams and
code examples with explanations for the testing doubles.

The first testing double described in the article are fakes.
The article provides the example of a SUT that depends on a Data Access Object
or DAO. In this case, a fake is used to replace the DAO so that the SUT can be
tested without the need of using a real database.

Stubs are the next testing double type discussed in the
article. Stubs are the simplest testing double type as they only provide canned
responses. These responses are predetermined and are used in place of actual

The last type in the article are mocks. Mocks are used for Behavioral
Testing. Mocks allow for testers to be able to verify the system is behaving
correctly, correct methods called at correct times. A simple case for a mock is
when a SUT depends on a method that does not return anything that can be
verified. The mock object will record when its methods are used, so that the
behavior can be checked.

I found the example diagrams in “Test Doubles — Fakes, Mocks
and Stubs,” by Michal Lipski, very helpful in understanding the differences of
the testing doubles. While this article does not go over two types of testing
doubles, dummies and spies, the three types that are discussed are described well.
Ultimately, when testing a system, it is imperative that the system can test
all its behaviors without relying on actual data.

From the blog CS@Worcester – D’s Comp Sci Blog by dlivengood and used with permission of the author. All other rights reserved by the author.

Mockito: Complicated at First Glance, Simple in Concept

Mockito is a mocking framework. In his article Mocks aren’t Stubs, Martin Fowler mentions that Mocks allow for behavior verification, rather than state verification, which verifies values after method calls to confirm the proper behavior occurred. Behavior verification instead makes sure the proper behavior happened, by confirming the proper methods themselves were called.

So mocking is a means of setting expectations of method calls on a mock object, and then verifying that those method calls actually happen. But how does this apply to Mockito?

Mockito is a bit more comprehensive. It has mocks, stubs and spies. It has a lot of methods, and a few different philosophies on how it should be used. Should everything be mocked except the system under test? Should you mock types that you didn’t create? When is partial mocking ok? These are questions that are not simply answered and will likely have answers that evolve over time. But the takeaway is that there is a lot to Mockito.

The best way to learn how Mockito can be used is to look at examples. Mockito’s documentation is a good resource and describes many of the concepts. However, the examples are a bit simple and isolated. A Vogella tutorial on Mockito has many better examples that shows how you can use Mockito in actual production code.

Complicated behavior is where mocking comes in handy. Stubs are great for making sure simple methods are called within objects. But in complicated systems, there is a lot more going on that may affect the behavior of the object under test. A mock object can trick external classes into believing that the Mock object is doing real work, by providing expected return values when a method is called. Furthermore, you cannot always check the state of an object without modifying the class itself, making behavior verification the only means of testing.

Mockito is commonly used in Android mobile app testing, especially when Android was officially using Java. In a mobile environment, you are building an app on top of a complicated framework, and as such you may encounter problems, or at least complications, when trying to test. A common example is the Context object in Android. It provides necessary information about the environment in which code is being run. Android Fragments and Activities might need this context internally, but the behavior of the Fragments and Activities should not depend on the actual Context object. Another issue arises: how do you set up application context in a test environment? With Mockito, these complications are resolved by simply creating a Mock Context object in the test cases. If you rely on return values within a mock object, you can even tell Mockito what it needs to return when methods are called.

Mockito can be used in combination with other test methods, using as many or as few of its features as desired. Mocking is quite possibly a necessity when testing complicated systems, and at the very least powerful and convenient.

From the blog CS@Worcester – Inquiries and Queries by ausausdauer and used with permission of the author. All other rights reserved by the author.

Seeing Doubles

In Computer testing there exist many ways to automate test and create ways to use objects that behave in a similar way to what they are supposed to be. In the blog post by Michal Lipski titled “Test Doubles — Fakes, Mocks and Stubs.”, he goes through the different doubles testing forms and explains their strengths and weaknesses and even shows diagrams and examples of code to help people better understand what each one of them is. Fakes being a working implementation that takes shortcuts and are different from the actual version. Stubs hold data given to the object via the tester to check if that data is being returned at the right time and calls. Mocks just write down the calls they receive and allow the tester to know that the actions were completed, but not if they were completed correctly. these forms of testing allow the tester to create version that satisfy conditions that they wish to check and understand.
I personally really enjoy the simplicity of these testing methods and how they don’t require much time or effort to be completed. while stubs are one of the simplest, in my opinion, they take a good amount of time and are thrown away later which makes them a hassle and a waste. Fakes seem to be a great choice for prototyping different things and allowing the tester to do as they please without any fear of damaging or changing any actual data. all together these testing methods are useful and will help me in my future in the computer science field.
link to the article referenced:

From the blog CS@Worcester – Tyler Quist’s CS Blog by Tyler Quist and used with permission of the author. All other rights reserved by the author.

Define-Use Testing

Many times in testing, the thing being checked is whatever state the object that is being returned is in. While this makes sense for testing whether the output is correct, there are often times you want to make sure that the correct variables are being defined and used. In define-use testing this is exactly what is done. In define-use testing, simple ordered pairs of the lines of code in which the object is created followed by the line the object is used, are created. I.E < 3, 5 >. This allows the user to make sure there is no unused objects that can take up memory for no reason. You can also follow the path of these objects to make sure they are being executed in proper order.

From the blog CS@Worcester – Journey Through Technology by krothermich and used with permission of the author. All other rights reserved by the author.

6 Weeks to Write 6 Blogs

Hello again my few readers, sorry I haven’t posted in a while but as you all know, I am a heavy procrastinator hence why I have 6 weeks to write 6 blogs in order to obtain an A in my class. Luckily, I am attending a presentation today so now I only have to write 5 blogs. Regardless, lets get this show on the road.

Today, I am going to talk about a blog with the title of “How Testers Can Become Agents of Change” Honestly, kind of a long title, but I don’t mind. The article is pretty short overall with 5 short sections. The first section is simple introduction to what the blog entry is going to be about. That is kind of like what I did for the first paragraph of this post. The author asks the reader to consider what they think is difficult at their job and how they will overcome said difficulty. Honestly, so far this sounds like one of those inspirational talks with some really outdated famous celebrity talking about how to be like them. This isn’t a bad thing, I just found that to be funny.

The post then explains how to identify opportunities for change. She mentions how there are three types of change: “Tool’s, processes, and people” (Norville) I assume this can mean that staff is always changing, methods are always changing, and tools are always changing which makes sense in a working environment. She says that anyone can bring change which sounds kind of cheesy, but I do like that because this shows that somebody like an Intern can bring about change. She then mentions a start, stop, continue model which is explained as the team discussing what they should stop doing, and what they should start doing. She uses an agile group as an example here and we learned about agile last semester.

Norville then talks about how to actually enact this change. This section is broken up into several subtopics which I think makes an article look cleaner and more organized. She tells the reader to know their audience in order to persuade them in the best way possible. She also says to fully understand the problem which is self explanatory. She describes the pitch as a way of presenting your information and evidence to the person you are trying to convince. This kind of follows the know your audience path. She also tells the reader to poke holes in the pitch. This is interesting because it is kind of exactly like testing where you try and break the code in order to ensure quality.

The last real paragraph is some tips that you can use in case you hit some bumps. The first of the tips is “Don’t Give Up” (This is in every inspirational post I swear). But regardless, it is true, even if you hit a bump along the way, you shouldn’t give up. The next tip is finding a team. This can be helpful with the knowing your audience part because everybody is different. She mentions learning from others successes, and I think this one is helpful because you can use methods that other people used to get your point across.

The actual last paragraph is just a closer telling the reader to start enacting change. I did enjoy this article, however this seems like an article that was directed at everyone rather than just testers. This isn’t a bad thing, but I would have liked to see more direct examples of how Software Testers can enact change. Overall this was a very well written blog. It was short and concise, and it was organized very well. I would recommend this article to anyone who is trying to make changes in their workplace, but they donf’t know how to start.

Making Software Quality a Critical Priority: How Testers Can Become Agents of Change

From the blog CS@Worcester – My Life in Comp Sci by Tyler Rego and used with permission of the author. All other rights reserved by the author.

If It’s Broke Don’t Fix it: Treating Bugs Like Buddha

For my last blog for this course instead of taking one
particular course subject, summarizing it, and theorizing about what actual
implementations may look like – I’d like to look at the whole course and do the
same. More specifically, I would like to cover how bugs or defects are actually
addressed, or not, in Software Development and Testing. To do this I have found
some interesting blog posts which argue that 100% of bugs may be able to be
fixed, but shouldn’t be. Instead, one should focus on serving the vast majority
of users under expected circumstances.

           Both blogs
focus on how impractical, and expensive, maintaining 100% stability or up-time
is. In the first focusing on bugs specifically they give many reasons why this
goal is unadvisable. The first is the prevalence of Agile Development,
dominating nearly the entire software development landscape. As such, the constraints
of this fast-paced development style limit the ability to do traditional QA testing;
if a program can have several revisions in a week, maybe even a day, then how
could a team reasonable test all these iterations. Instead, the author suggest
a stability monitoring tool to automatically test each revision.

addition, they suggest that eliminating 100% of bugs would eliminate many which
users would never see, so why waste resources addressing them? Even if you
could fix everything how could you possibly know? This focus is reinforced when
considering the truly incredible breadth of devices that one may have consider,
and specifically in mobile development: where they cite the over 24,000
different android devices on the market. One must focus on the average expected
user experience and not waste time fussing with the outliers until, presumably,
a bug report is filed.

The second blog discusses
defects in systems in a similar way, covering more or less the same points, although
mentioning a rather obscure possibility: being legally challenged for claiming
that your product has no defects. Instead, what I believe they are trying to emphasize
is that products are always fallible, and the amount of resources required to
get them even close is impossible or impractical. As a result, as we move
forward in this class and towards graduation I think we should resist the impulse
to try for complete perfection and instead focus on what is achievable and
provides the best experience for the majority of users.


Not all bugs are worth fixing and that’s okay
The Zero Defect Fallacy

From the blog CS@Worcester – Press Here for Worms by wurmpress and used with permission of the author. All other rights reserved by the author.

Path Testing

Testing comes in many forms and all are important in their own regard. Path testing is a type of testing used to determine that each path a program can take will produce the correct output. By doing this you are able to determine if there is a fault in the code and where this fault is produced. With path testing you are able to create decision to decision paths or DD paths and you can use these to create a program graph. A DD path is a node in a program that satisfies a certain condition, such as it has one in-degree and one out-degree, meaning there is only one flow of code that can happen that enters the node then leaves. When creating all these nodes they can be assembled into a program graph. These graphs help show the flow of the program and can help with determine how to test your code with path testing.

From the blog CS@Worcester – Journey Through Technology by krothermich and used with permission of the author. All other rights reserved by the author.

Clarifying Edge Testing

I happened upon a very
interesting blog recently, searching for more examples of edge case testing,
wherein the author provides an interesting set of common edge cases. While they
use the same names or similar ones as those we’ve seen lately in class some
have new meaning here. In addition, they may have an actual value tied to the
name – such as zero, one, and two – but their values are defined otherwise.

 In order they
are as such:

  • Zero “represents
    any form of null input”, including the actual value zero but also null, an empty
    List or Array, et cetera. This of course is to test the capability of a program
    to deal with input that isn’t properly usable by it.
  • One represents what
    we have referred to as nominal up to this point, meaning a valid normal input
    which should test proper functionality of a program under ideal conditions.
  • Two does not
    correspond to the value two, but rather refers to testing the same code twice,
    usually in sequence, to see how repeated executions affect a system.
  • Two to Max-1 is
    most like one, in that it represents a nominal value as well, but in opposition
    this value should not be the absolute simplest needed to function but an average
    use case; meaning possibly complicated.
  • Max is
    fairly self-explanatory, used to test the upper limit accepted by a program, and
    can sometimes be an extreme value. As such, it can test the limits of the
    program under incredible load.
  • Max + 1 is
    used to ensure that limits placed on an application are working and that anything
    that does not correspond to a valid range is rejected in a reasonable manner.

These would most closely correspond to Normal Boundary Value Testing as we have covered in class but they are each less abstract than those counterparts. They provide an insight on what these values look like in actual QA testing, as well as expectations upon being used. Two for example does not have an actual value associated with it, but rather refers to a testing orthodoxy outlined above. One example of a test in this vein is found in another blog, in which the author proposes the edge case of the same user trying to log in from two different computers. I believe between the two of these blogs a clearer picture of the concepts we have covered can be found through these more concrete examples of the testing procedure.


A Beginner’s Guide to Testing: Error Handling Edge Cases
Build Strong Edge Test Cases

From the blog CS@Worcester – Press Here for Worms by wurmpress and used with permission of the author. All other rights reserved by the author.