Monthly Archives: November 2016

Exploratory Testing

This week I was reading a book by James A. Whittaker called Exploratory Software Testing. The book, obviously, is about exploratory software testing.

The second chapter starts with a quote by Alan J. Perlis: There are two ways to write error-free programs: only the third one works. Basically he means that there is no way to write error free programs!

So if there is no way to write error free programs, is there a way to prevent bugs in programs? He goes on to discuss that all bug prevention techniques are generally developer-oriented like writing better specs, performing code reviews, running static analysis tools and performing unit tests. He argues that all of these techniques suffer from some fundamental issues: the “developer makes the worst tester” problem, the “software at rest” problem, and the “no data” problem. The “software at rest” problem brings to light the fact that code reviews and static analysis techniques try to test programs when there are at rest (not running). The problem with this approach is that most bugs only surface when they are running. What is the point of testing software when you cannot identify bugs?

The author goes on to argue that manual testing is more powerful than automated testing. Manual test techniques like exploratory testing allows the full power of the human brain to be used on finding bugs. This chapter is a prelude to the rest of the book which discusses the methods and wisdom used for exploratory testing.

From the blog Software Testing – The blog about software by Sudarshan and used with permission of the author. All other rights reserved by the author.

Why You Should Be Writing Descriptive Test Names

A non descriptive test name will cause anyone unfamiliar with the tests to be forced to read through the whole test code to figure out what is actually being tested. An example of this coming from the article is this code along with the test name:

@Test public void isUserLockedOut_invalidLogin() {
  authenticator.authenticate(username, invalidPassword);
  assertFalse(authenticator.isUserLockedOut(username));

  authenticator.authenticate(username, invalidPassword);
  assertFalse(authenticator.isUserLockedOut(username));

  authenticator.authenticate(username, invalidPassword);
  assertTrue(authenticator.isUserLockedOut(username));
}

Looking at this test the test name “isUserLockedOut_invalidLogin” is not descriptive enough considering what the test does. This causes us to have to look through the code and figure out that it locks the user out after 3 failed login attempts. A more appropriate test name would be “isUserLockedOut_lockOutUserAfterThreeInvalidLoginAttempts”. This test name tells us exactly whats going on and having to read through the code is not needed. The scenario and the expected outcome should be in the test name. This will save many hours by just having to read the test name instead of reading through code to figure out what the test is doing.

https://testing.googleblog.com/2014/10/testing-on-toilet-writing-descriptive.html

From the blog CS@Worcester – Software Testing by kyleottblog and used with permission of the author. All other rights reserved by the author.

Unit Tests Vs End-to-End Tests

I have made a post previously containing what End-to-End tests were and there advantages and disadvantages. In this post I will compare End-to-End testing to Unit testing. Unit testing is done by taking a small area of an application and isolate it and then perform tests. This produces fast reliable tests.

With end-to-end testing you must wait until the software is complete before any tests can be created. The test results tend to be unreliable, and if a bug is found it is not known where it was found. The advantages of unit testing outweigh the advantages of end-to-end testing.Unit testing is fast, reliable and can find failures easily. End-to-end testing is slow, not very reliable and failures are hard to find, the only benefit to end-to-end testing is that it can more effectively simulate a real user. But that alone does not outweigh unit testings advantages.

https://testing.googleblog.com/2015/04/just-say-no-to-more-end-to-end-tests.html

From the blog CS@Worcester – Software Testing by kyleottblog and used with permission of the author. All other rights reserved by the author.

Don’t Panic, It’s Just A Sensitive Database (Week 10 11/25/16)

There are many testers in your department, but only a few (in some cases only a very few) are in charge of the heart of all life in the creation everyone is pouring their figurative sweat and blood and a ton of coffee, the database.

Now, first and foremost, you should be proud of yourself. You got the golden ticket from your superior being tasked to defend and maintain what is keeping the company standing with the potential of chaos if something goes horribly wrong, but that’s why you are there to help prevent such a disaster from happening! You are the one that maintains the server contain confidential, very important information (in most cases  a lot of it), but the good news is it’s not as bad as you think!

continue-testing.jpg

To communicate with the database, you can simply just write queries in order to talk to the monstrous amount of data. However, if you really want to mess with it because you want to ensure no threats will emerge from here on out, then you would have to make a duplicate server that will not completely destroy the entire company which gives you the perfect opportunity to do whatever you want to it in order to avoid actual destruction of the real database itself!

From using White Box Testing or Black Box Testing, you may need to have more experience with the database to promote effective maintaining of the sensitive information. You have to know the ins and outs of the database as well as becoming the database if you really want to feel a connection of how it behaves. By all means, you can start with database auditing and monitoring. But it’s a long list just to fit in this one post.

Source: http://quality-assurance-software-testing.blogspot.com/2009/03/database-testing.html

From the blog CS@Worcester – Dan's Tech Rant by danbarbara and used with permission of the author. All other rights reserved by the author.

Are Testers People Who Can’t Code? (Week 9 11/18/16)

In the world of software development, there are the software programmers and there are the testers. No, there are more than those in the software field but the two that work together like peanut butter and jelly on a sandwich are the ones that code and bring the program to life and the ones that make sure it stays living by making it not fail and fall.

The bottom line is testers are testers because either they wanted to be or ended up in life as a tester, as well as the possibility that programming did not work for them. Same for the software developers, they are who they are because they want to be or either they ended up in that life. In the end, the programmers and testers need each other because one can not work without the other. Sure Testers can be ‘second class citizens’ who are people who can not code well, i’m one of those people. I’m sure that it doesn’t meant that those are the only type of people who are software testers.

quote-testing-proves-a-programmer-s-failure-debugging-is-the-programmer-s-vindication-boris-beizer-80-81-67

Kind of how Kualitatem states, software testers and programmers are hammer and chisel. Not to mention they both equally have challenging jobs that requires a lot of man power. Its kind of like politics with the Democrats and Republicans. Sure they both have different views, but if only one major party exists the outcome of our country would change forever due to one common set of beliefs. One needs the other in order to keep the balance and as well survive.

(source: http://www.kualitatem.com/blog/are-testers-failed-programmers)

From the blog CS@Worcester – Dan's Tech Rant by danbarbara and used with permission of the author. All other rights reserved by the author.

A Tester’s Guide to the Messaging Galaxy (11/11/16 Week 8)

Ever since the dawn of man kind (and possibly before it) communication became the sticks and stones of our interaction with one another where one tower is supported with the other. Now the same applies many centuries later from throwing mud at each others faces and using smoke signals to the effective use of typing on your computer’s keyboard using the magic of electronic capabilities, innovative programs, and of course the internet.

As software tester, you communicate with your team like how your team communicates to you. Your team relies on your findings and looks forward to you, the tester, on keeping them informed in the most effective way possible. For the best possible way of communicating with your team through bug reports and status updates, the most important aspect is to make sure your method of communication allows the repentant to get the best quality from your intended message as possible.

test-evaluation-report

If you are looking forward to improving your communication skills for your testing job, then some tips that would help range from how you structure your reports to asking your self questions of what you are addressing and what the other end would want to read or not receive more can be found here. .

Source: http://www.softwaretestinghelp.com/improve-written-communication-skill-as-a-qa-tester/

From the blog CS@Worcester – Dan's Tech Rant by danbarbara and used with permission of the author. All other rights reserved by the author.

How Good Are Your Test Cases?

An important and on-going question that QA testers always ask is how good their test cases are. This can be measured in many ways; one way being code coverage. Code coverage is a good method to check if all parts of your code was executed; although it still would not tell much of how good your tests are even if your program has good code coverage.

An alternative to test how good your test cases are is through mutation testing. This was an interesting read for me because I had just read an article that briefly touched upon mutation testing last week so it was a cool follow-up article. Mutation testing is a technique that is used to assess how well your test cases react to faults that are intentionally injected into your code. This can be useful to evaluate how efficient your test cases can handle those faults and how it reacts to any unexpected issues.

Article: http://www.softwaretestingmagazine.com/videos/evaluating-test-cases-quality-with-mutation-testing/?utm_source=feedburner&utm_medium=feed&utm_campaign=Feed%3A+SoftwareTestingMagazine+%28Software+Testing+Magazine%29

From the blog CS@Worcester – Tan Trieu's Blog by tanminhtrieu and used with permission of the author. All other rights reserved by the author.

Should We Value UI Automation Tests?

Karlo Smid wrote an interesting piece on the value of UI automation testing.

Lets begin by defining what UI automation testing is. According to Microsoft,

Automated tests that drive your application through its user interface (UI) are known as coded UI tests (CUITs). These tests include functional testing of the UI controls. They let you verify that the whole application, including its user interface, is functioning correctly. Coded UI Tests are particularly useful where there is validation or other logic in the user interface, for example in a web page. They are also frequently used to automate an existing manual test.

So what are the values of UI Automation Tests?

  • They represent executable documentation for your product.
  • Assist in answering questions about complicated business scenarios.

Requirements for Good UI Testing?

  • Language that enables you to write less code.
    • Example: Ruby
  • Page object pattern implementation
  • The use of Jenkins which is an integration tool
  • The use of rspec for the testing framework
  • Smart regression testing

From the blog CS@Worcester – My Blog by justcodeit94 and used with permission of the author. All other rights reserved by the author.

11 Things You’re Doing Wrong When Testing Code

Full Article by Matt Lacey

In this post, Matt Lacey discusses 11 things that developers do wrong when creating and testing their code.

The first one he lists is that developers can just fall into the trap of just not making any tests for their code. He states that a way to fix this is to just start creating tests for the code that you’re already working on and plan to always do this for future projects.
The next item his list is that developers need to design and make tests when they start their projects. This makes it easier to implement into the code that is currently being worked on rather than adding in tests later. Also adding in tests later may require developers to make architecture changes and will add on additional time.
The third item on the list is writing failing tests. Lacey goes into explaining that your should not specifically make tests to fail (or have a red outcome) because the point of writing tests is to make sure that our system is working properly.
The fourth thing on the list is having a fear of unimplemented tests. Having these types of tests (which also have a proper names for the functions they are preforming) will serve as a reminder about the intended function of the system as well as what the test should accomplish.
Fifth on the list is simply not giving your tests good, identifying names. Lacey states that the way you name your tests isn’t important; what matters is that your naming behaviors are consistent, and properly describe what the tests are suppose to do.
The next item is making tests that do too much. Here he gives a reminder that your tests should only be testing one thing; this could even be a series of simple things that make up one task even, but it should not be testing multiple long tasks.
Seventh on Lacey’s list of what not to do is not testing you code. Lacey points out that creating mocks and running them does not actually mean you are testing the code. He states that these types of “tests” could just end up being a waste of time and effort.
The next thing on the list is worrying too much about code coverage. Lacey states the knowing “how much of the code is executed when the tests are run should be useful but because it doesn’t consider the quality of the tests that are executing the code it can be meaningless.” He goes on to say that code coverage is only of interest if the coverage is too high or too low. Lacey then address what you kind of code should be tested then by stating “Does the code contain non-trivial complexity? If it does then you need some tests. If it doesn’t then you don’t.”
The ninth item that Lacey views as a mistake is only doing one type of testing. He states that if you’re only making one type of tests then you are not properly testing your software. It takes a multiple different tests to find mistakes in all areas of your system.
Number ten on the list focusing too much on short term tests. Tests should not be made to test something that is static; like if the code is written correctly. Instead tests should be written the make sure that the code continues to function correctly over time. Tests should be able to be run repeatedly to make sure that no new errors occur if new code is added or changed.
The last item on Lacey’s list of what not to do is being a developer that relies on someone else to create and run tests. Tests should always be available for developers to run if the need arises. If there is something that will hinder a quick execution of tests then that will just prevent developers from running the tests they need. The best thing to do, according to Lacey, is to make sure that “code and tests…be kept in the same place and any setup needed should be scripted.”

From the blog CS WSU – Techni-Cat by clamberthutchinson and used with permission of the author. All other rights reserved by the author.

Testing IoT Devices: Key Areas

Full Article by Pavel Novik

In this article, Pavel Novik talks about the phenomena known as the Internet of Things (IoT), and goes over the importance for companies to properly tests the software that is used by these products.

Pavel Novik points out that many companies use IoT products, but more than half of them “do not have a software testing strategy for this specific aspect of their IT infrastructure.” However being able to test certain devices can be an issue, Novik points out, if the company only sells the application for running the devices because they might not be able to have all the products on hand to tests. Novik suggests that companies that are in this position to find out which devices their customer base buy the most, and then limit their testing to those products.

Novik then goes on to talk about what areas should be tested when it comes to IoT devices. The first one that he suggests is security; making sure that no one is able to steal personal information from the devices, and also prevent attacks from happening which may lead to accidents. The second area is performance; this is to make sure that all the data that is being sent between these products without fail even when some sort of disruption takes place. The third area Novik points out is functionality; this is testing “to verify that the application functions meet all the functional requirements,” and that testers should “strive to go beyond general test scenarios, and consider body movements, voice commands and sensor utilization when writing test cases.” The fourth and last area that’s cover is compatibility; covering this area while testing will make sure the product works with other versions, devices, and systems that it you may encounter further down the line.

From the blog CS WSU – Techni-Cat by clamberthutchinson and used with permission of the author. All other rights reserved by the author.