Category Archives: Software Testing

How Google Deals with “Flaky Tests”

Read the Full Article

This post, by John Micco, talks about how the Software Developers at Google deal with and minimize the damage of “Flaky Tests”. Micco defines a flaky test “as a test that exhibits both a passing and a failing result with the same code.” He goes on to state that this causes major problems because how difficult it is to find the cause of the flaky tests and the frequency of which these kinds of tests appear. There is also the issues of a flaky tests being dismissed only to find out later that it was a real failure which sometimes results in passing through broken code.

There are several strategies that the developers at Google use to minimize the damage and confusion that is caused by these flaky tests. One way is to only report a true failure if the test fails three times in a row. This, however, can be costly if the test that is being run takes a large amount of time to complete. Another way is that they implement a tool that monitors the flakiness of a test. If the tool determines that the flakiness for a test is too high, it removes the test from the critical path. The last strategy they have is to use another tool that monitors flakiness levels. When the levels change, the tool tries to find the reason for the test’s change in flakiness.

From the blog CS WSU – Techni-Cat by clamberthutchinson and used with permission of the author. All other rights reserved by the author.

When to stop!

This week I read an article called When to Stop Testing (Exit Criteria in Software Testing). The article talks about when to stop testing a software product.

No matter how much we test we can never be absolutely sure that the product is bug free. There is no methods that we can use to prove that a software product is absolutely bug free. The only thing that we can do is iteratively test for all possible defects. However, testing costs money. So when do we stop? Do we stop when we run out of money? Run out of time? Have reach a certain number of bug count? Have fixed a certain number of bugs?

The following are the factors to consider when deciding to stop testing.

Testing can be stopped when:

Requirements:

  • 100% Requirements coverage is achieved.

Defects:

  • Defined / Desired Defect count is reached.
  • All Show Stopper defects or Blockers are fixed and No known Critical / Severity 1 defect is in Open Status.
  • All High Priority defects are identified and fixed.
  • Defect Rate falls below defined acceptable rate.
  • Very few Medium Priority defects are open and have a workaround in place.
  • Very few low priority open defects that do not impact software usage.
  • All High Priority defects are re-tested and closed and corresponding Regression scenarios are successfully executed.

 Test Coverage:

  • Test Coverage should be 95% achieved.
  • Test case Pass Rate should be 95%. This can be calculated by formula
    • ( Total No of TCs Passed / Total number of TCs ) * 100.
  • All critical Test cases are passed.
  • 5% Test cases can be failed but the Failed Test cases are of low priority.
  • Complete Functional Coverage is achieved.
  • All major functional / business flows are executed successfully with various inputs and are working fine.

Deadlines:

  • Project Deadline or Test Finish deadline is reached.

Test Documents:

  • All Test Documents / deliverables (Example – Test Summary Report) are prepared, reviewed and published across.

Budget:

  • Complete Testing Budget is exhausted.

 

 

From the blog Software Testing – The blog about software by Sudarshan and used with permission of the author. All other rights reserved by the author.

Flexibility = ???

Admit it, we as the people want technology to do everything for us. People want something that can play music, download movies, track fitness levels, helps concentration, schedule appointments, socialize, shop online, and complete assignments simultaneously for them all in one app. Now an app like this doesn’t exist (at least I don’t think it does…) but imagine the flexibility the app would possess if it did. Ultimately, when we think of qualifications for “good technology” , flexibility is a key requirement.

When it comes to software testing the qualifications are mutual. The goal is for test suites to be flexible; meaning it is adaptable in any given condition and is able to correctly handle any given situation that may be presented to it. This seems great and all but if you ask me, the greater the flexibility is, the greater the complexity becomes.

To test whether or not this is true: assertTrue(flexibility.equals(complexity);

 

From the blog CS@Worcester – Tan Trieu's Blog by tanminhtrieu and used with permission of the author. All other rights reserved by the author.

The Pros of Platform-led Testing

Platform-led Testing by Lakshminarasimhan Rajabather

In this article, Rajabather goes over the positives of using platform-led testing to help with automation across all the stages of the software development life cycle. The first advantage that he lists for using this approach is that it cuts down on the cost and time to use assurance across the development’s life cycle. This is also positive to businesses since their goal is to minimize cost and time, and maximize quality.

Rajabather points out is that platform-led testing makes sure that the software is constantly being checked and validated at every stage of development. This is able to be done because, as previously stated, it is now possible to have software assurance constantly being implemented through the entire life cycle. Platform-led testing also works well with Agile development because it “promises quick sprints, rapid sign-offs and a measurable transfer of value from one sprint to the next.”

The last two advantages of platforms that Rajabather lists show that they are beneficial to businesses. He states that platforms allow businesses to build upon both industry and third party analytical tools which makes it so that the tools can be customized to fit a certain need. The last benefit is that platforms are not “restricted only to the requirements, design or execution stage of the lifecycle.”

Please read the full article for more information on this subject.

From the blog CS WSU – Techni-Cat by clamberthutchinson and used with permission of the author. All other rights reserved by the author.

Git

This week I was reading a book call Learn Git in a Month of Lunches by Umali. The book was very good in explaining how Git works and is written in a very easy to understand way. My understanding of Git has improved by a lot after reading this book. I would recommend this book to anyone who wants to understand Git.

From the blog Software Testing – The blog about software by MegaMind and used with permission of the author. All other rights reserved by the author.

Introduction

Hello,

My name is Ryan. I am a senior in computer science at Worcester State University. This blog will be used to discuss various topics in the field of computer science which I find interesting.

From the blog CS@WSU – :(){ :|: & };: by rmurphy12blog and used with permission of the author. All other rights reserved by the author.

Do NOT Manually Test

With the increasing fast development of technology in today’s society, you can almost guarantee that 99% of people have some type of advanced device on them at all times. For those of you who are not included in that 99%, you may like doing things on your own without having to rely on technology to do so for you, which includes manual testing.

While manual testing is still active today, it is not very practical or time-efficient. Us as humans are imperfect which leads to the chance to make mistakes with manual testing, not to mention the amount of hours it will take depending on the scale of the program.

An obvious solution to this issue is to replace manual testing with automation testing. Not only will this save you the trouble and time of having to set up tests that require complex requirements and pre-conditions but it is also helpful in inputs of data that need multiple fields of information.

 

Other 5 Blogs Read:

1. http://www.testingtv.com/2016/09/21/from-junit-to-mutation-testing/
2. http://www.softwaretestingmagazine.com/knowledge/oblique-testing/?utm_source=feedburner&utm_medium=feed&utm_campaign=Feed%3A+SoftwareTestingMagazine+%28Software+Testing+Magazine%29
3. http://www.softwaretestingclub.com/profiles/blogs/why-manual-tests-are-error-prone-and-what-to-do-about-it
4. http://www.softwaretestingmagazine.com/news/blazemeter-acquire-by-ca-technologies/?utm_source=feedburner&utm_medium=feed&utm_campaign=Feed%3A+SoftwareTestingMagazine+%28Software+Testing+Magazine%29
5. http://www.testingtv.com/2016/08/30/code-quality-in-practice/

From the blog CS@Worcester – Tan Trieu's Blog by tanminhtrieu and used with permission of the author. All other rights reserved by the author.

Introduction

My name is Andon and I am a student at  Worcester State University with major Computer Science. In this blog, I will be sharing my ideas about software testing, software development, and big data. I believe that software development is a very interesting field to be explored!

From the blog Table of Code by Andon S and used with permission of the author. All other rights reserved by the author.

Introduction

My name is Andon and I am a student at  Worcester State University with major Computer Science. In this blog, I will be sharing my ideas about software testing, software development, and big data. I believe that software development is a very interesting field to be explored!

From the blog Table of Code by Andon S and used with permission of the author. All other rights reserved by the author.

Introduction

My name is Andon and I am a student at  Worcester State University with major Computer Science. In this blog, I will be sharing my ideas about software testing, software development, and big data. I believe that software development is a very interesting field to be explored!

From the blog Table of Code by Andon S and used with permission of the author. All other rights reserved by the author.