Exit Criteria

Exit Criteria, Software Quality, and Gut Feelings

Imagine a situation, within the last two weeks the company has had eight new builds to combat every bug that had been found. Release day is tomorrow, currently there are no critical bugs due to not having a chance to retest, but technically this passes the exit criteria…..A simple gut feeling will say that it is a terrible idea to go live, but it is not your choice. How do we avoid such situations?

Create Comprehensive Exit Criteria

The exit criteria can’t be a simple bug count; it has to be sophisticated. It has to take into account the number of critical bugs discovered over the last test cycle, check that the count of new bugs has a lowering rate of appearance at x bugs/week, y days since last critical bug, etc. Looking at the overall development timeline can help better estimate if you are going to meet your exit criteria and allow your team to be more proactive.

 

https://www.stickyminds.com/article/exit-criteria- software-quality- and-gut- feelings

From the blog CS@Worcester – Kyle Polewaczyk by kpolewaczyk and used with permission of the author. All other rights reserved by the author.

Mutation Testing

This week I decided to play around with mutation testing and the pitest system for mutation testing.

So what is mutation testing? The idea is to mutate your code and check to see if your tests still pass. If your tests fail that means that your tests were able to detect the mutant. In mutation testing terms this means that the mutant was killed. The quality of your tests can be gauged from the percentage of mutations killed.

If the test cases pass after the mutation, that may indicate an issue with the test suite. The whole idea of mutation testing is to test the effectiveness of the tests.

I decided to run the pitest tool on the tictactoe example I found on the Test-Driven java Development book by Alex Garcia.

I have attached a link to the code and the corresponding pitest results (https://drive.google.com/drive/folders/0B586m1QhLSUXQzBlWHVZNkxnMmc?usp=sharing). The mutation coverage was 95% and all the mutations were killed. I wonder why all of the mutations were killed?

From the blog Software Testing – The blog about software by Sudarshan and used with permission of the author. All other rights reserved by the author.

Functional Testing(Week 8)

Functional Testing is a testing prerequisite for seeing how precisely the system functions. Each part of a system must be examined to make sure they are working separately and alongside each other. It is essential to guarantee the quality of the software. 

Some Types of functional testing are Unit testing, which is to make sure each part of the code is able to deliver the desired output, black box testing which is when you do not know how the code works but are testing  crucial functions, regressing testing which is used to ensure the new code has not introduced any new bugs, system integration testing which determines the operability and compatibility of the software with other applications, and user acceptance testing which factors in the user experience and how well they interact with the system. 

There are lots of functional testing tools out there that you can use. Selenium, cucumber and HP QTP are examples of tools you can use. Regressing testing is absolutely necessary to ensure your software is working how it is supposed to be working. 

Article I used : 

http://www.cigniti.com/blog/functional-testing-need-know-right-now/

 

Other Articles i looked at: 

http://www.softwaretestingtricks.com/2012/04/top-5-common-myths-in-software-testing.html

http://www.cigniti.com/blog/5-reasons-agile-testing-software-development-go-hand-hand/

http://www.cigniti.com/blog/3-challenges-of-implementing-devops-testing-how-to-deal-with-them/

http://www.cigniti.com/blog/what-is-the-role-of-qa-in-the-world-of-devops/

 

From the blog CS@Worcester – Site Title by jonathanpaizblog and used with permission of the author. All other rights reserved by the author.

How to Overcome 5 Software Testing Traps (Week 7)

If you have gone down the software testing path, and have been doing it for some time , chances are you have run into several different problems.

One of the problems you may have run into is running out of test ideas. One of the easiest ways to overcome this problem is by working with someone else. Working with someone else allows you to bounce ideas off each other, and expand on each other’s ideas. So this may help reduce the risk of you running out of test ideas.

Another problem you may run into is missing the testing goal. If you have a set of goals that you have to meet up to, and there is something holding you back from achieving this, whether it be not understanding completely, or not well defined goals, this may happen to you. To overcome this, you need to ask relevant questions to make sure you know exactly what is expected from you.

Another problem is if you are suffering from an in- attentional blindness. Basically if you miss something that is very simple. To fix this you should change the focal length of your testing approach. Keep and eye open for anything else that might be going on during test execution. Explore other areas that are not mentioned in your test case.

Have you ever come across an error or bug that you didn’t report as an error or a bug because you were not sure if they were an actual bug? This can be fixed by trusting your testers instinct. Follow your instinct, if you think its a bug, it probably deserves your attention and very well may be a bug. It is a much better approach then you ignoring it and pretending it is not a bug.

What to test and what can be skipped? It may be overwhelming by the number of possibilities and choices to approach testing. As time goes on testing goes on, tests get more and more complex.

Chances are you will run into these problems at some time in your career. Now that you have a solid understanding of what they are and how to overcome them, you should be able to deal with them efficiently 

Article I used:

http://www.softwaretestingtricks.com/2014/08/top-5-software-testing-mistakes-solutions.html

Other Articles i looked at:

http://www.softwaretestingtricks.com/2013/08/Importance-of-Cost-effective-Software-Testing.html

http://www.softwaretestingtricks.com/2013/05/best-cloud-based-test-management-tool-qtest.html

http://www.softwaretestingtricks.com/2013/03/agileload-enterprise-load-stress-performance-testing-tool.html

http://www.softwaretestingtricks.com/2013/02/best-load-stress-performance-testing-tool-WAPT.html

From the blog CS@Worcester – Site Title by jonathanpaizblog and used with permission of the author. All other rights reserved by the author.

Code Review

I cannot stress how important it is to double check your work. During my last Software Quality Assurance and Testing class, we had the pleasure of doing a code review exercise. The tasks were to review code from a well known game called Sir Tommy Solitaire. It was a group exercise, but the day before class we were given the assignment to review the code and to try and find any faults within the program.

At first, I didn’t even look at the code, I just wanted to see how the program ran. By doing this, I had a glimpse about the coder and if he/she thought about user experience. I tried to type in random things just to see how the program would handle my inputs. After that I would make a note of what I thought wasn’t right and did the same when I looked at the code.

This in class experience was very insightful, because it shows you how to collaborate as team and shows you the importance of writing elegant, clean and efficient code. If you wrote a program and it works, but you did not document it at all, the program won’t be very good, because no one will know what your methods do, or why you have a certain class or named a variable the way you did, because there is no documentation. Documentation is very imperative in the programming realm, because there are different styles of code, and documenting things makes it easier for other developers to know what you are doing.

From the blog CS@Worcester – My Blog by justcodeit94 and used with permission of the author. All other rights reserved by the author.

Exploring Software

Imagine you are giving a large software system to test, how do you test the system? Where do you start? Chapter 4 of the exploratory software testing book (by James Whittaker) answers this question.

The author uses the metaphor of the tourist. Suppose you visit a city like London. How do you go about touring the city? Do you walk around? Use the bus? Use a guide? There are many ways to travel a city and so does exploring a large software system. There has to be some sort of structure to help us achieve our goals.

The following are the goals of exploratory testing:
1) Gain an understanding of how an application works, what its interface looks like and what functionality it implements.
2) To force the software to exhibit its capabilities.
3) To find bugs.

The author develops a tourist guidebook for software to help us explore software. The author separates software functionality into overlapping districts as follows:
1. business district (The main part or “where stuff gets done” in the software systems)
2. Historical district (Legacy code)
3. Tourist district (Many cities have districts where only tourists go. Software is similar in                                       that novice users will use certain features that experienced users would                                        not.)

4.  Hotel district (Every tourist needs a place to rest. Software is actually quite busy when it                                     is “at rest”, and testing tours need to test those features)
5. Seedy district (Places where illegal and bad stuff happen. Seedy tours find places of                                              vulnerability in the software)

The author goes on to explain how to test each of these “districts” and what kind of bugs they might release.

Tours is mechanism to organize a tester’s thinking about how to approach exploring an application and in organizing actual testing. The tours help testers make the myriad decisions about which path to choose, inputs to apply, or parameters to select.

From the blog Software Testing – The blog about software by Sudarshan and used with permission of the author. All other rights reserved by the author.

Four main functional tests to ensure software quality (WEEK 11)

These post is about a collection of four most recommend tests that a developer must have knowledge. According to info-Tech report there are four functional tests that can ensure software quality. The six functional tests are unit test, regression test, system integration, tests  and acceptance test. 


To ensure every line of code executes correctly is recommender to use Unit Testing, Unit Testing is the process of testing each unit of code  in a single component. One unit is the smallest part in a software, one unit can be a single program or function.  


To ensure every function produces its expected outcome developers have to use Functional Testing. Functional testing indicates concerns about the correct decision of functional requirements. Functional testing is like a black box testing which don’t require any knowledge of the underlying implementation.


To ensure that all of the functions combine to deliver the desired business results developers have to use System testing. This type of testing executes end to end functional tests that can cross software units, in order to help become fully aware of the goals of assure that components deliver the desired business results. 


To ensure new changes did not adversely affect other parts of the system developers should use Regression testing. Regression  testing is used to ensure code modifications that have not accidentally introduced bugs into the system. When using Regression  testing is recommended to include plans from the original unit and system tests phases which can show that existing functionality behaves as planed. 


I wrote in my previous blogs posts about these four of these functional tests i really recommend you to read my previous posts because there you can find dip details about unit testing and etc.




Citation


http://searchsoftwarequality.techtarget.com/report/Six-functional-tests-to-ensure-software-quality






From the blog Table of Code by Andon S and used with permission of the author. All other rights reserved by the author.

Four main functional tests to ensure software quality (WEEK 11)

These post is about a collection of four most recommend tests that a developer must have knowledge. According to info-Tech report there are four functional tests that can ensure software quality. The six functional tests are unit test, regression test, system integration, tests  and acceptance test. 


To ensure every line of code executes correctly is recommender to use Unit Testing, Unit Testing is the process of testing each unit of code  in a single component. One unit is the smallest part in a software, one unit can be a single program or function.  


To ensure every function produces its expected outcome developers have to use Functional Testing. Functional testing indicates concerns about the correct decision of functional requirements. Functional testing is like a black box testing which don’t require any knowledge of the underlying implementation.


To ensure that all of the functions combine to deliver the desired business results developers have to use System testing. This type of testing executes end to end functional tests that can cross software units, in order to help become fully aware of the goals of assure that components deliver the desired business results. 


To ensure new changes did not adversely affect other parts of the system developers should use Regression testing. Regression  testing is used to ensure code modifications that have not accidentally introduced bugs into the system. When using Regression  testing is recommended to include plans from the original unit and system tests phases which can show that existing functionality behaves as planed. 


I wrote in my previous blogs posts about these four of these functional tests i really recommend you to read my previous posts because there you can find dip details about unit testing and etc.




Citation


http://searchsoftwarequality.techtarget.com/report/Six-functional-tests-to-ensure-software-quality






From the blog Table of Code by Andon S and used with permission of the author. All other rights reserved by the author.

Four main functional tests to ensure software quality (WEEK 11)

These post is about a collection of four most recommend tests that a developer must have knowledge. According to info-Tech report there are four functional tests that can ensure software quality. The six functional tests are unit test, regression test, system integration, tests  and acceptance test. 


To ensure every line of code executes correctly is recommender to use Unit Testing, Unit Testing is the process of testing each unit of code  in a single component. One unit is the smallest part in a software, one unit can be a single program or function.  


To ensure every function produces its expected outcome developers have to use Functional Testing. Functional testing indicates concerns about the correct decision of functional requirements. Functional testing is like a black box testing which don’t require any knowledge of the underlying implementation.


To ensure that all of the functions combine to deliver the desired business results developers have to use System testing. This type of testing executes end to end functional tests that can cross software units, in order to help become fully aware of the goals of assure that components deliver the desired business results. 


To ensure new changes did not adversely affect other parts of the system developers should use Regression testing. Regression  testing is used to ensure code modifications that have not accidentally introduced bugs into the system. When using Regression  testing is recommended to include plans from the original unit and system tests phases which can show that existing functionality behaves as planed. 


I wrote in my previous blogs posts about these four of these functional tests i really recommend you to read my previous posts because there you can find dip details about unit testing and etc.




Citation


http://searchsoftwarequality.techtarget.com/report/Six-functional-tests-to-ensure-software-quality






From the blog Table of Code by Andon S and used with permission of the author. All other rights reserved by the author.

Four main functional tests to ensure software quality (WEEK 11)

These post is about a collection of four most recommend tests that a developer must have knowledge. According to info-Tech report there are four functional tests that can ensure software quality. The six functional tests are unit test, regression test, system integration, tests  and acceptance test. 


To ensure every line of code executes correctly is recommender to use Unit Testing, Unit Testing is the process of testing each unit of code  in a single component. One unit is the smallest part in a software, one unit can be a single program or function.  


To ensure every function produces its expected outcome developers have to use Functional Testing. Functional testing indicates concerns about the correct decision of functional requirements. Functional testing is like a black box testing which don’t require any knowledge of the underlying implementation.


To ensure that all of the functions combine to deliver the desired business results developers have to use System testing. This type of testing executes end to end functional tests that can cross software units, in order to help become fully aware of the goals of assure that components deliver the desired business results. 


To ensure new changes did not adversely affect other parts of the system developers should use Regression testing. Regression  testing is used to ensure code modifications that have not accidentally introduced bugs into the system. When using Regression  testing is recommended to include plans from the original unit and system tests phases which can show that existing functionality behaves as planed. 


I wrote in my previous blogs posts about these four of these functional tests i really recommend you to read my previous posts because there you can find dip details about unit testing and etc.




Citation


http://searchsoftwarequality.techtarget.com/report/Six-functional-tests-to-ensure-software-quality






From the blog Table of Code by Andon S and used with permission of the author. All other rights reserved by the author.