5 Common Mistakes in Automated Testing

Original Article

In his article, Rohit Sharma goes over the top five most common mistakes users have made when using automation tools for testing.

The first item on his list is when testers “think of automation testing as little more than record and replay.” He goes on to say that proper automation testing is done when a tester has customized the generated script for their specific needs. Sharma also mentions that the record feature should only be seen as an outline and not as the finished script.

Second on the list is when testers do not validate the scripts that they are using. Sharma suggests that using checkpoints, at as many places as possible, can help to remedy this situation.

The third thing is that you don’t want to just validate your visible components when testing. Anything that happens on the back end should also be checked to make sure that your script is properly validating it (the two examples that Sharma gives are communications to a database, and making sure a file was automatically created properly).

Fourth on the list is making sure to remember that automated testing cannot replace human testing. Sharma states that “automation is a great way to augment your testing efforts,” but automation testing only works well “when you know what to look for.” In the case that he states, you could have a web page that generates unreadable text, but, unless you have someone looking out for that, you are unlikely to find it with automation testing.

The last item that Sharma talks about is making creating “inappropriate test cases.” He points out that there are multitudes of paths that a user can use in your application so trying to test all of them would waste time. Instead, try to figure out how to “condense all possible paths to a small sample of highly representative test cases,” so that it covers the majority of your user’s activity.

 

From the blog CS WSU – Techni-Cat by clamberthutchinson and used with permission of the author. All other rights reserved by the author.

Mutation Testing

Mutation Testing

It is one thing to use the many techniques to come up with a set of test. There is software that will go through you code and test cases and provides a measure which show how much of your code is actually covered by the tests. This is a great start but how do you know if your tests are really helpful. That is to say are do the test do a good job at testing the code and would they pick up on errors. To really judge how strong your test suite is you need to be able to test your test. One way of doing this is by purposely making changes to your code and seeing if your tests catch it. This is called mutation testing.

Manually going through your code would be very time consuming and tedious. The benefits would not be worth the effort. Luckily there are programs that automate this process. I personally looked at Pit Mutation Testing. It can be found at pitest.org.  In the case of Pitest it works with Java and the software directly modifies the byte code which makes the process much faster as you don’t need to edit the source then recompile.

The code systematically works through the code and makes changes, these changes are referred to as mutants. It then runs the tests to see if these mutants survive or are killed. It then generates a report that shows what lines are covered by the test as well as what mutations it generated. For each mutation it will show if it survived or was killed. In the case of a surviving mutation it will indicate whether it was tested and survived or if it was simply never tested.

Some of the mutation it tries include changing logical operators, changing the the operations in equations and returning null.

these changes include changes logical statements.

From the blog CS@WSU – :(){ :|: & };: by rmurphy12blog and used with permission of the author. All other rights reserved by the author.

7 Best Practices for Agile Test Driven Development

Original Article

This article goes over seven different practices that will help with Test Driven Development when using the Agile approach.

The first thing that should be done is make sure you “avoid functional complexity.” The point of this type of approach is to keep it simple. The article suggests going over with your team to make sure that the test covers all the functionality that is needed.

The second thing is to make sure you have a clear understanding on what it is you are trying to achieve. It’s advised that you follow standard documentation and naming for your tests. This is so that developers are able to come back to it and understand what was the intended purpose for the set of tests.

The third item is that code is not overly complected. You want to make sure that your test is kept simple so that the code is easily readable and has room for any necessary improvements.

Fourth on the list is to make sure you are testing everything repeatedly. Always run tests before and after coding, and after refactoring since you never know when adding something new might break your tests. Also remember that when you refactor, to make sure that the node is still “maintainable and adheres to standards.”

Fifth point that is made is to “maintain code sanctity.” This means that you should be using version control tools to keep your code in check; which is extremely import if you have multiple developers working on the same code.

The sixth thing is making sure that your team is aware and understands how the application works. To do this make sure that the system documentation is clear and that all team members understand how the system and tests work. This will not only make sure that when new code is added, the overall program will not break, but also that new tests will work in the first run.

The last item that the article points out is to “know when to use TDD.” Test Driven Development is not suppose to be used for any testing that will take a long time. Always remember TDD primary function is to be used for projects that can be tested quickly.

From the blog CS WSU – Techni-Cat by clamberthutchinson and used with permission of the author. All other rights reserved by the author.

4 Warning Signs That Your Beta Testing Process Is in Danger

Original Article

In this article, Ran Rachlin goes over the four warning signs that may signal that your current beta testing process might be in trouble.

The first point that he talks about is if your beta testing is having huge delays. The main way Rachlin says to over come this obstacle is to set strict deadlines for the team (this includes testers). He also points out to make sure that the objectives for the deadline are clear and reasonable; this is to make sure that everyone is on the same page and no one gets worried about what needs to be done. The other thing that this step includes is to make sure that constant contact is made with the testers so that communication is not lost and you receive better feedback.

The second item Rachlin touches upon is making sure that you don’t underestimate how much time it will take to go through testing. If you are on a tight deadline to get your product out, he recommends a few things that you should do. One is to make sure you have experienced tests for your product. Two is to stay in contact with the testers (as mentioned above), and inform them of the “time crunch” they are dealing with. These steps will help make sure that the testers know what they are doing before hand and are aware of the challenges they need to overcome.

The third thing is to be aware of the well being of the testers you’ve hired. If you’re testers become frustrated with the product they are working on, this can hinder the speed of testing the application. Again it helps to keep contact with the testers to make sure they are on track with the work, but you also want to make sure that they feel appreciated for the work they are doing; especially if they are in a time crunch. It may also help to give incentives near the end of a deadline to encourage testers to finish early or for a job well done. Although you should always have backup plan just in case things become too much for the testers that you have.

The last point that Rachlin makes is to ask yourself “Are we testing the target market and devices?” He says that you must make sure of two things; that you make sure the testers you have are from the target market and that you are using the “most popular devices and carriers in this target market” for your tests. If all of these issues are addressed, you should have minimal issues with they beta testing process for your app.

From the blog CS WSU – Techni-Cat by clamberthutchinson and used with permission of the author. All other rights reserved by the author.

Coverage Testing

One of the trickiest things about software testing is knowing when we have tested enough. We could spend hours testing the software and believe that we have done a good job and later find have a bug show up that was triggered by an input that we did not think to test.

One way that we could quantify the amount of testing done is to use a coverage metric. The coverage metric is a score that based on the number of functions tested in the source code.

I used the EclEmma code coverage plugin for Eclipse and I used the code from my previous blog. The following was the output of the code coverage.

 

capture

One thing that I noticed was that the coverage metric only tests to see if a particular method was traversed by a test case and not whether the test case passes. One of the test cases could have failed and the test coverage metric would stay the same. Obviously!

From the blog Software Testing – The blog about software by Sudarshan and used with permission of the author. All other rights reserved by the author.

Bash Scripts

After reading many articles this week, I found one that poked my interest. It was an article about bash scripts. It was quite interesting because it was a tutorial on how you can use a bash script that will allow you to find all of the files that uses a particular method.

Karlo Smid shows us how you do it.

screen-shot-2016-12-14-at-11-28-00-am

 

screen-shot-2016-12-14-at-11-28-41-am

How it works

  • Reads item from the txt.file
  • Each row value is searched using grep utility
  • Recursive search in all folders
  • Outputs files that contains those searched items

Bash scripting is very useful because you can just tell the computer what to do automatically when you run it rather than having to do everything manually. When you run the script that you want, it improves time and also efficiency because it can do what you’re trying to do very quickly.

Read More

From the blog CS@Worcester – My Blog by justcodeit94 and used with permission of the author. All other rights reserved by the author.

May The Humor Be With You (Week 12 12/9/16)

“Developer: There is no I in TEAM.  Tester: We can’t spell BUGS without U”  – Dima Al-Adham

At the 12th week approaches, this post will cover what I think is the most important aspect of Software Testing (even development in general) which is to be funny! Yes I know, you may be confused of how being funny adds on to being a Software Tester. Its understandable that there are not any humorous people out there as well and the fact that they even try to be funny might completely turn against them. But as you may –hopefully — have noticed, each of these blog posts are written in a way with a touch of unique personality, right?

” Why did you test it that way? The users should never do that.”  – Herbert L Warren

In the real world, reading a blog full off information about a topic could be boring and can give you stress if you are not able to skim through it since you may be looking for specific information. People who read whatever is written needs to feel welcome and know that everything that they are experience was created by a human being with their own personality. In the end we are not automated robots! In software testing, take a look in the code of the project that you are working on. Take a look at the comments made in the files, does it seems like a robot wrote those to you? When you are making reports that seems stressful and dull to read, do you think your supervisor wants to have the impression that they have robots working for them? In the end of the day we are all human, and if possible a little bit of laughter, even when something is slightly different, is enough to put a smile on someone’s face when they are repeating their everyday job.

“To tell somebody that he is wrong is called criticism. To do so officially is called testing.”  – Gaurav Khurana

Just remember you are a human being, and not just a software developer or a plain worker. In the end of the day, no matter what we go through whether its creating projects or reading a bunch of students blogs?, at the end of the day humor is everywhere and can lift the spirits of everyone, like I hope I did when I wrote 12 weeks off somewhat different blog posts with a touch of who I am.

Source: http://www.mindfultester.com/adding-humour/

From the blog CS@Worcester – Dan's Tech Rant by danbarbara and used with permission of the author. All other rights reserved by the author.

Making Your Own Rules When There Isn’t Any.(Week 11 12/2/16)

There may come a time in your life as a software tester, where you do not know what to test. The guided path that your leader should have instructed you to base of your testing skills off of are completely blank and un-given. What do you do? The answer is simple, you do not take this moment as a stress-full event in your life where you may think one wrong, un-sure move is going to ruin your entire career, but take it as a opportunity so a blank slate with no requirements can turn into a free-style session and let your bug breaking mind run free!

Are you not given what to specifically test for and a strict deadline to go as the cherry? While, you could ask your supervisor for a list of requirements since they have not created them or misplaced the information to give you by accident, but chances are they do not have any and want you to go wild!

But how do you go wild when it comes to testing? Well, when the key phrase “oh i do not know what i am testing for” comes around, as a tester a familiar key term should pop in your head and that is the key term Black Box Testing! You can go crazy finding errors and bugs in a project and by the time the project is done, you have flexed your creativity muscles but enriching the curious experience of being a tester. The best of Black Box Testing can be found here.

 

Source: https://testzius.wordpress.com/2016/12/13/testing-without-requirements/

From the blog CS@Worcester – Dan's Tech Rant by danbarbara and used with permission of the author. All other rights reserved by the author.

On bug prevention

I really enjoyed this blog. The main theme more or less was
can bugs be prevented? He goes on to talk about how in testing we go at the
software from many different angles looks for bugs, but just how often is it
that we actually think about preventing them and are they actually inevitable
and can they even be prevented in the first place? While bugs are inevitable he
believes that they can be somewhat prevented and after reading his blog I
believe he is right.
He goes on to explain the general things we can do about
prevention such as better communication, trying to make less mistakes in the
code, understanding the platform that is being worked on and the list goes on.
The main question he asks is this; What can a tester do to help with bug
prevention? Testing happens usually after the bug is already in place and it is
the then too late to prevent the bug right? Well maybe not. He goes on to say
that the testing and the results report can actually influence the teams
thought in some areas or things. He says, “For
example, something as ‘innocent’ as an email saying: ‘I’m planning some testing
of feature X and I wanted to make sure I’m not duplicating work on this. What
kind of unit tests have you done on this?’ can gently nudge coders to think
about unit-testing their code (and the bug prevention benefits that come from
writing code that is unit-testable in the first place).
I really enjoyed this blog as he
makes many good points and if I am ever in the tester role I will surely take
some of his points to heart.
You can read the full blog here:

https://offbeattesting.com/2016/12/13/preventing-bugs/

From the blog format c: /s by c-braley and used with permission of the author. All other rights reserved by the author.

On bug prevention

I really enjoyed this blog. The main theme more or less was
can bugs be prevented? He goes on to talk about how in testing we go at the
software from many different angles looks for bugs, but just how often is it
that we actually think about preventing them and are they actually inevitable
and can they even be prevented in the first place? While bugs are inevitable he
believes that they can be somewhat prevented and after reading his blog I
believe he is right.
He goes on to explain the general things we can do about
prevention such as better communication, trying to make less mistakes in the
code, understanding the platform that is being worked on and the list goes on.
The main question he asks is this; What can a tester do to help with bug
prevention? Testing happens usually after the bug is already in place and it is
the then too late to prevent the bug right? Well maybe not. He goes on to say
that the testing and the results report can actually influence the teams
thought in some areas or things. He says, “For
example, something as ‘innocent’ as an email saying: ‘I’m planning some testing
of feature X and I wanted to make sure I’m not duplicating work on this. What
kind of unit tests have you done on this?’ can gently nudge coders to think
about unit-testing their code (and the bug prevention benefits that come from
writing code that is unit-testable in the first place).
I really enjoyed this blog as he
makes many good points and if I am ever in the tester role I will surely take
some of his points to heart.
You can read the full blog here:

https://offbeattesting.com/2016/12/13/preventing-bugs/

From the blog format c: /s by c-braley and used with permission of the author. All other rights reserved by the author.