Category Archives: Software Testing

7 Best Practices for Agile Test Driven Development

Original Article

This article goes over seven different practices that will help with Test Driven Development when using the Agile approach.

The first thing that should be done is make sure you “avoid functional complexity.” The point of this type of approach is to keep it simple. The article suggests going over with your team to make sure that the test covers all the functionality that is needed.

The second thing is to make sure you have a clear understanding on what it is you are trying to achieve. It’s advised that you follow standard documentation and naming for your tests. This is so that developers are able to come back to it and understand what was the intended purpose for the set of tests.

The third item is that code is not overly complected. You want to make sure that your test is kept simple so that the code is easily readable and has room for any necessary improvements.

Fourth on the list is to make sure you are testing everything repeatedly. Always run tests before and after coding, and after refactoring since you never know when adding something new might break your tests. Also remember that when you refactor, to make sure that the node is still “maintainable and adheres to standards.”

Fifth point that is made is to “maintain code sanctity.” This means that you should be using version control tools to keep your code in check; which is extremely import if you have multiple developers working on the same code.

The sixth thing is making sure that your team is aware and understands how the application works. To do this make sure that the system documentation is clear and that all team members understand how the system and tests work. This will not only make sure that when new code is added, the overall program will not break, but also that new tests will work in the first run.

The last item that the article points out is to “know when to use TDD.” Test Driven Development is not suppose to be used for any testing that will take a long time. Always remember TDD primary function is to be used for projects that can be tested quickly.

From the blog CS WSU – Techni-Cat by clamberthutchinson and used with permission of the author. All other rights reserved by the author.

4 Warning Signs That Your Beta Testing Process Is in Danger

Original Article

In this article, Ran Rachlin goes over the four warning signs that may signal that your current beta testing process might be in trouble.

The first point that he talks about is if your beta testing is having huge delays. The main way Rachlin says to over come this obstacle is to set strict deadlines for the team (this includes testers). He also points out to make sure that the objectives for the deadline are clear and reasonable; this is to make sure that everyone is on the same page and no one gets worried about what needs to be done. The other thing that this step includes is to make sure that constant contact is made with the testers so that communication is not lost and you receive better feedback.

The second item Rachlin touches upon is making sure that you don’t underestimate how much time it will take to go through testing. If you are on a tight deadline to get your product out, he recommends a few things that you should do. One is to make sure you have experienced tests for your product. Two is to stay in contact with the testers (as mentioned above), and inform them of the “time crunch” they are dealing with. These steps will help make sure that the testers know what they are doing before hand and are aware of the challenges they need to overcome.

The third thing is to be aware of the well being of the testers you’ve hired. If you’re testers become frustrated with the product they are working on, this can hinder the speed of testing the application. Again it helps to keep contact with the testers to make sure they are on track with the work, but you also want to make sure that they feel appreciated for the work they are doing; especially if they are in a time crunch. It may also help to give incentives near the end of a deadline to encourage testers to finish early or for a job well done. Although you should always have backup plan just in case things become too much for the testers that you have.

The last point that Rachlin makes is to ask yourself “Are we testing the target market and devices?” He says that you must make sure of two things; that you make sure the testers you have are from the target market and that you are using the “most popular devices and carriers in this target market” for your tests. If all of these issues are addressed, you should have minimal issues with they beta testing process for your app.

From the blog CS WSU – Techni-Cat by clamberthutchinson and used with permission of the author. All other rights reserved by the author.

Coverage Testing

One of the trickiest things about software testing is knowing when we have tested enough. We could spend hours testing the software and believe that we have done a good job and later find have a bug show up that was triggered by an input that we did not think to test.

One way that we could quantify the amount of testing done is to use a coverage metric. The coverage metric is a score that based on the number of functions tested in the source code.

I used the EclEmma code coverage plugin for Eclipse and I used the code from my previous blog. The following was the output of the code coverage.

 

capture

One thing that I noticed was that the coverage metric only tests to see if a particular method was traversed by a test case and not whether the test case passes. One of the test cases could have failed and the test coverage metric would stay the same. Obviously!

From the blog Software Testing – The blog about software by Sudarshan and used with permission of the author. All other rights reserved by the author.

Mutation Testing

This week I decided to play around with mutation testing and the pitest system for mutation testing.

So what is mutation testing? The idea is to mutate your code and check to see if your tests still pass. If your tests fail that means that your tests were able to detect the mutant. In mutation testing terms this means that the mutant was killed. The quality of your tests can be gauged from the percentage of mutations killed.

If the test cases pass after the mutation, that may indicate an issue with the test suite. The whole idea of mutation testing is to test the effectiveness of the tests.

I decided to run the pitest tool on the tictactoe example I found on the Test-Driven java Development book by Alex Garcia.

I have attached a link to the code and the corresponding pitest results (https://drive.google.com/drive/folders/0B586m1QhLSUXQzBlWHVZNkxnMmc?usp=sharing). The mutation coverage was 95% and all the mutations were killed. I wonder why all of the mutations were killed?

From the blog Software Testing – The blog about software by Sudarshan and used with permission of the author. All other rights reserved by the author.

Exploring Software

Imagine you are giving a large software system to test, how do you test the system? Where do you start? Chapter 4 of the exploratory software testing book (by James Whittaker) answers this question.

The author uses the metaphor of the tourist. Suppose you visit a city like London. How do you go about touring the city? Do you walk around? Use the bus? Use a guide? There are many ways to travel a city and so does exploring a large software system. There has to be some sort of structure to help us achieve our goals.

The following are the goals of exploratory testing:
1) Gain an understanding of how an application works, what its interface looks like and what functionality it implements.
2) To force the software to exhibit its capabilities.
3) To find bugs.

The author develops a tourist guidebook for software to help us explore software. The author separates software functionality into overlapping districts as follows:
1. business district (The main part or “where stuff gets done” in the software systems)
2. Historical district (Legacy code)
3. Tourist district (Many cities have districts where only tourists go. Software is similar in                                       that novice users will use certain features that experienced users would                                        not.)

4.  Hotel district (Every tourist needs a place to rest. Software is actually quite busy when it                                     is “at rest”, and testing tours need to test those features)
5. Seedy district (Places where illegal and bad stuff happen. Seedy tours find places of                                              vulnerability in the software)

The author goes on to explain how to test each of these “districts” and what kind of bugs they might release.

Tours is mechanism to organize a tester’s thinking about how to approach exploring an application and in organizing actual testing. The tours help testers make the myriad decisions about which path to choose, inputs to apply, or parameters to select.

From the blog Software Testing – The blog about software by Sudarshan and used with permission of the author. All other rights reserved by the author.

Four main functional tests to ensure software quality (WEEK 11)

These post is about a collection of four most recommend tests that a developer must have knowledge. According to info-Tech report there are four functional tests that can ensure software quality. The six functional tests are unit test, regression test, system integration, tests  and acceptance test. 

To ensure every line of code executes correctly is recommender to use Unit Testing, Unit Testing is the process of testing each unit of code  in a single component. One unit is the smallest part in a software, one unit can be a single program or function.  

To ensure every function produces its expected outcome developers have to use Functional Testing. Functional testing indicates concerns about the correct decision of functional requirements. Functional testing is like a black box testing which don’t require any knowledge of the underlying implementation.

To ensure that all of the functions combine to deliver the desired business results developers have to use System testing. This type of testing executes end to end functional tests that can cross software units, in order to help become fully aware of the goals of assure that components deliver the desired business results. 

To ensure new changes did not adversely affect other parts of the system developers should use Regression testing. Regression  testing is used to ensure code modifications that have not accidentally introduced bugs into the system. When using Regression  testing is recommended to include plans from the original unit and system tests phases which can show that existing functionality behaves as planed. 

I wrote in my previous blogs posts about these four of these functional tests i really recommend you to read my previous posts because there you can find dip details about unit testing and etc.


Citation

http://searchsoftwarequality.techtarget.com/report/Six-functional-tests-to-ensure-software-quality





From the blog Table of Code by Andon S and used with permission of the author. All other rights reserved by the author.

Mutation Testing (WEEK 10)

The term mutation in software testing stand for modifying the test code and can check the existing test and detect or reject the change mutes. Mutation in Java is an additional class with a single modification compared to the original code, for example it can be the change of a logical operator  in a if section :

if( a && b ) {...} => if( a || b ) {...}

When detecting or rejecting a modification by the existing test in marked as killing a mutant. If we have a perfect test code no class mutant can survive. Testing with mutant testing can be very expensive so that is one reason why is better to write a perfect code. There are tools to create mutants and run all the tests against each of the mutants automatically. The mutation creation is based on mutation operators that are which can show typical programming errors.

This picture shows a report which list all the mutant killed by ruing the mutation testing.
Here are some of the benefits when we apply the mutation testing:

Can show all new errors to the developer
Can detect hidden defects in the structure of code
Can increase customer satisfaction index because the product can be less buggy.
Maintaining and debugging the product can be easier.


Mutation testing is very useful way to find errors and fix them and it can speedup the proses of finding bugs in the code.


Citation
http://www.codeaffine.com/2015/10/05/what-the-heck-is-mutation-testing/

From the blog Table of Code by Andon S and used with permission of the author. All other rights reserved by the author.

First Software Technical Review

This week in our Software Testing class; we were assigned the task of software technical reviewing some code that was provided by our professor. It was quite exciting since that was the first legitimate testing in a group I have done in my programming career. The concept of software technical review so fairly simple and straight to the point so it was not difficult to dissect what it was that we needed to do in order to effectively review the given code. The outline of the assignment was to first individually review the code by ourselves without discussing our results with other team members. Afterwards, you would then meet up as a whole team and discuss about what each other found in terms of errors and faults within the code. The designated recorder of the group records any changes made during the meeting and compiles together a master list of all of the members’ combined discovery.

Although the testing process itself was new and exciting, there were a couple of complaints I had. The only factor that made the assignment frustrating is that some team members did not individually review the code before the meeting. Due to this, instead of delegating time to discuss our findings, we spent most of the class time waiting for some members to look over the code so we barely had any time to actually discuss as a whole group. Besides that however, it was a fun and memorable new experience in which I learned a lot!

From the blog CS@Worcester – Tan Trieu's Blog by tanminhtrieu and used with permission of the author. All other rights reserved by the author.

Deadly Linux Commands

When it comes to Linux, I would say I know the overall basics of how it is ran, operated and how to use it but I certainly am not a professional of knowing the more advanced commands and functions that Linux has to offer. Knowing that Linux is a very useful OS, especially for programmers, I wanted to know a little bit more about its commands. I was browsing through a couple of sites, blogs and articles and there was one that immediately catch my attention. The title of the article was “Deadly Commands on Linux” (http://www.softwaretestingclub.com/profiles/blogs/deadly-commands-on-linux) and it talked about 6 commands that are dangerous in the sense that it could completely wipe out your system if it is unintentionally (or intentionally) used. To briefly demonstrate just how dangerous these commands could be, there is a command rm -rf / – that will delete any specific file on a hard drive and all connected devices or if you want, delete all the files on a hard drive and connected devices. That’s not even the worst part; the worst part about this command is that it does not even ask you if you are sure about deleting those files, it will do it without prompting you about it. To be honest, I thought that command was pretty cool but on the other hand, I’m just thinking to myself “Why would you even have a command like that in the first place…”. But hey, after reading this article, at least now I know which commands to avoid when working with Linux!

From the blog CS@Worcester – Tan Trieu's Blog by tanminhtrieu and used with permission of the author. All other rights reserved by the author.

Exploratory Testing

This week I was reading a book by James A. Whittaker called Exploratory Software Testing. The book, obviously, is about exploratory software testing.

The second chapter starts with a quote by Alan J. Perlis: There are two ways to write error-free programs: only the third one works. Basically he means that there is no way to write error free programs!

So if there is no way to write error free programs, is there a way to prevent bugs in programs? He goes on to discuss that all bug prevention techniques are generally developer-oriented like writing better specs, performing code reviews, running static analysis tools and performing unit tests. He argues that all of these techniques suffer from some fundamental issues: the “developer makes the worst tester” problem, the “software at rest” problem, and the “no data” problem. The “software at rest” problem brings to light the fact that code reviews and static analysis techniques try to test programs when there are at rest (not running). The problem with this approach is that most bugs only surface when they are running. What is the point of testing software when you cannot identify bugs?

The author goes on to argue that manual testing is more powerful than automated testing. Manual test techniques like exploratory testing allows the full power of the human brain to be used on finding bugs. This chapter is a prelude to the rest of the book which discusses the methods and wisdom used for exploratory testing.

From the blog Software Testing – The blog about software by Sudarshan and used with permission of the author. All other rights reserved by the author.