Category Archives: Software Testing

How Good Are Your Test Cases?

An important and on-going question that QA testers always ask is how good their test cases are. This can be measured in many ways; one way being code coverage. Code coverage is a good method to check if all parts of your code was executed; although it still would not tell much of how good your tests are even if your program has good code coverage.

An alternative to test how good your test cases are is through mutation testing. This was an interesting read for me because I had just read an article that briefly touched upon mutation testing last week so it was a cool follow-up article. Mutation testing is a technique that is used to assess how well your test cases react to faults that are intentionally injected into your code. This can be useful to evaluate how efficient your test cases can handle those faults and how it reacts to any unexpected issues.

Article: http://www.softwaretestingmagazine.com/videos/evaluating-test-cases-quality-with-mutation-testing/?utm_source=feedburner&utm_medium=feed&utm_campaign=Feed%3A+SoftwareTestingMagazine+%28Software+Testing+Magazine%29

From the blog CS@Worcester – Tan Trieu's Blog by tanminhtrieu and used with permission of the author. All other rights reserved by the author.

Selenium IDE

This week I decided to play a bit with the selenium IDE. So what is selenium IDE? Selenium IDE was developed to allow testers and developers to record their actions as they follow the workflow that they need to test.

The Selenium IDE is basically an add-on to the Firefox browser. You could download it from http://seleniumhq.org/download/. Once it is installed, you can use Alt+Ctrl+s to start selenium.

Once the selenium IDE is open, you could enter the URL of the website you want to test and could click on record. The IDE will now record every action you make on the website. Once you are done, you could stop the recording and click on the play entire test suite button on the IDE  and Selenium would now replay everything you did.

From the blog Software Testing – The blog about software by Sudarshan and used with permission of the author. All other rights reserved by the author.

Levels of testing (Week 8)

There are different levels in a testing  process. Levels of testing include different methodologies which can be used during the process of software testing. There are two main levels of testing : functional testing and non-functional testing. One of my previews posts talks about black-box testing, functional testing is a type of black-box testing which is based on the specifications of the software that has to be tested. The software can be tested by providing input and then the results are examined in order to conform to the functionality which it was meant for. Functional testing of a software is conducted on a complete, integrated system to evaluate the system’s compliance with its specified requirements. 
Functional testing includes five steps : 
Step 1 The determination of the functionality that the intended application is intended to perform.
Step 2 The creation of test data based on the specifications of the application.
Step 3 The output based on the test data and the specifications of the application.
Step 4 The writing of test scenarios and the execution of test cases.
Step 5 The comparison of actual and expected results based on the executed test cases.
In order to have an effective testing process all the above steps have to be followed and applied to the testing policies of every organization for this reason it will make sure that the organization maintains the strictest of standards when it comes to software quality. This part is about testing an application from its non-functional attributes. Non-functional testing includes testing a software from the requirements which are nonfunctional.  Performance, Security, User interface are some requirements examples of non-functional testing. Performance testing is one important and mandatory testing type in terms of the following: Speed, Capacity, Stability and Scalability. Performance testing can be either qualitative or quantitative. Security testing includes testing a software in order to identify any gaps from security.


Security testing should ensure the below list of features: 

Confidentiality, Integrity, Authentication, Availability, Authorization, Non-repudiation, Software data is secure, Software is according to all security regulations, Input checking and validation, SQL insertion attacks, Injection flaws, Session management issues, Cross-site scripting attacks, Buffer overflows vulnerabilities and Directory traversal attacks.
The levels of testing are important because in order to meet the functional and technical specifications  we have to follow all the steps and features that I listed above. 

Citation
https://www.tutorialspoint.com/software_testing/software_testing_levels.htm

From the blog Table of Code by Andon S and used with permission of the author. All other rights reserved by the author.

Hack More!

I read an article this week about which programming language hackers use (http://www.softwaretestingclub.com/profiles/blogs/which-programming-language-do-hackers-use) and while at first, I didn’t see the relationship between hacking and software testing, I came to realize that they have a good amount of similarities. Thinking more in depth about it, the act of hacking is just finding vulnerabilities in a software or program which is the same objective as QA testers (although the intents may be different).

Something interesting about the article was that when asked, “what programming language do hackers use”, the answer given by an expert hacker Adrian Lamo was totally not what I was expecting. He responded that he does not use a programming language which caught me off-guard because I didn’t think it was even possible to hack / test without the use of a programming language. The most memorable and significant piece of information I got out of this article was when Adrian Lamo further explains his answer by saying that its more important to learn how to read code rather than learning how to code. By learning how to efficiently read code, you can dissect how programmers think, what their tendencies are, and what common mistakes they make which can give you a better idea of what to test.

From the blog CS@Worcester – Tan Trieu's Blog by tanminhtrieu and used with permission of the author. All other rights reserved by the author.

Mocking

The purpose of a unit test is to verify an assumption about the behavior of a software system. And unit tests must be automated so that any assumptions can be verified quickly.

A unit test should also exhibit the following characteristics:

  1. It should have fast test execution.
  2. A test should not depend on database access or any long running tasks. A test should isolate external dependencies.

 

A Mock Object is an object that substitutes for a real object. Mock objects are simulated objects that mimic the behavior of real objects. Mock objects allow you to set up test scenarios without using large resources like databases. Instead of calling a database for testing, you can simulate your database using a mock object in your unit tests. This makes tests run faster.

 

 

 

From the blog Software Testing – The blog about software by Sudarshan and used with permission of the author. All other rights reserved by the author.

Can test automation replace human testers (Week 7)

This article lists the pros and cons for test automation  and human testers.  This days the IT managers and directors are in dilemma because they have to decide what is better for their department to use for software testing the automated testing or the manual testing. Do they have to hire a team of software testers or would they be better using automated testing? These strategies have big difference. Both these strategies have pros and cons here are some:

Automated Testing

Pros: The main advantage of using automated testing is the speed, IT managers can use this strategy to test the code faster then manual testing and also they can re-use the automated testing tool many times that they need. Automated testing can be used ion different operating systems. It is very effective for build verification and testing.

Cons: A down side of automation testing is not helpful with UI testing, it can not be trusted all the time also it has high initial cost of tools.

Manual Testing

Pros: Manual Testing is the testing of software for bugs in the software system. Testers execute test cases without using any automation tools. Lets talk about some of the pros, companies that can’t afford automated test tools can use testers to write and execute test cases, this strategy is flexible compared to automation testing.

Cons: Manual testing can be very slow compared to automation testing. The tester have to be engaged 100% with the software and has to know the structure of it. If the tester don’t know the structure of the software can be a high chance of missing out on some bugs.

Citation
http://www.thinksys.com/can-test-automation-replace-human-testers.shtml

From the blog Table of Code by Andon S and used with permission of the author. All other rights reserved by the author.

When to Mock?

Full Post by Uncle Bob

In this blog post, Uncle Bob goes over the basics when you should use mocks, and the advantages and disadvantages of using them.

He first goes over what will happen to your test suite if you have no mocks (he’s also going under the assumption that this is for a large application). The issues that arise when no mocks are used is that, during execution, your tests could take from several minutes to several hours to run. The next problem that could happen when no mocks are used is that your test suite might not cover everything. Bob states that “error conditions and exceptions are nearly impossible to test without mocks that can simulate those errors.” The example he gives for this is if you have files or databases tables that need to be deleted, this task my be to dangerous without the use of mocks. The last thing Uncle Bob brings up is that tests are really fragile if you don’t use mocks. This is because tests can easily be disrupted by faults in the system that are not even caused by what you’re testing.

This may seem that mocks are always the way to go, but Uncle Bob covers what happens when you use too many mocks (for this Uncle Bob goes under the assumption that you have mocks between all classes in your test suite). This first issue is that “some mocking systems depend strongly on reflection,” and if you have too many this can slow down your testing phase. The other thing is that, if you use mocks between all classes, you may end up making mocks that create other mocks. This can create two problems. One, the set-up code you’re using can end up getting convoluted. Two, this can lead your “mocking structure [to] become tightly coupled to implementation details causing many tests to break when those details are modified.” The last thing Bob points out is if you have too many mocks, you may end up have to create extra interfaces “whose sole purpose is to allow mocking,” and this can lead to “over-abstraction and the dreaded ‘design damage’.”

This may make it seem that using or not using mocks can lead to issues, so what is the correct why to use mocks in your test suite? Uncle Bob states that the method he uses for this is to only “mock across architecturally significant boundaries, but not within those boundaries”; this means to only mock when it comes to using any external software or anything that is rather large, like a database or web server. This way of mocking will address most of, if not all, the issues that came up when having no mocks or too many. The other thing that Uncle Bob suggests doing is to always create your own mocks and not rely on other software tools to do it. He points out that most mocking tools have their own language that can become time consuming to learn. Also creating your own mocks maybe more beneficial in the end because it forces you to name your mocks and put them in directories which allows you to use them for other tests. The other benefit to creating your own mocks is that it makes you take time to create your own mocking structure.

In the end, Bob suggests using mocking as little as possible and to strive to design tests to not use mocks.

From the blog CS WSU – Techni-Cat by clamberthutchinson and used with permission of the author. All other rights reserved by the author.

Improving Testing Skills

I read an article this week entitled “How to Write a Test Case Structured for Your Project and Your Team” (http://blog.testlio.com/how-to-write-a-test-case-structured-for-your-project-and-your-team) which at first I did not find fascinating because as Computer Science majors, most of us should already be familiar with the general structure of test cases. However, despite the initial lack of interest I gave the article a chance and skimmed through its content and it turns out that there were some interesting test case structures that I have never heard or though of before.

One in particular that caught my attention was the “Verify-using-with-to test cases”. The article outlines the type of user of this structure to be beginning testers or employees (which is basically us once we graduate!) This test style can be applied to projects or any magnitude and is mainly focused on the actual language of the tests. The reason for the naming of this structure is because it follows the 4 key words: verify, using, with, and to. Instead of going into the definition of these 4 key terms, the example used in the article is this.

Verify: Courses send to bookstore
Using: Student ID, “Look up books”
With: Student is registered in at least one course
To: Show student courses list in student bookstore

After reading the article and going over the examples provided, I did find this particular test structure to be not only useful, but pretty easy to use so it’s something I look forward to using the next time I test!

From the blog CS@Worcester – Tan Trieu's Blog by tanminhtrieu and used with permission of the author. All other rights reserved by the author.

Unit Testing with Mock Objects (Week 6)

This article is posted by IBM and explains how the Mock Object can be used. In this article we will find some of the benefits of using Mock Objects for writing Unit tests for objects that act as mediators. Unit testing has become widely accepted for software development. When you write an object you must also provide an automated test class containing methods that put the object through its paces, calling its various public methods with various parameters and making sure that the values returned are appropriate.When we’re dealing with simple data or service objects, writing unit tests is straightforward. However, many objects rely on other objects or layers of infrastructure. When it comes to testing these objects, it is often expensive, impractical, or inefficient to instantiate these collaborators. For example, to unit test an object that uses a database, it may be burdensome to install, configure, and seed a local copy of the database, run your tests, then tear the local database down again.A mock object conforms to the interface of the real object, but has just enough code to fool the tested object and track its behavior. For example, a database connection for a particular unit test might record the query while always returning the same hardwired result. As long as the class being tested behaves as expected, it won’t notice the difference, and the unit test can check that the proper query was emitted.

Here are some of the common coding style for testing with mock objects:

  • Create instances of mock objects
  • Set state and expectations in the mock objects
  • Invoke domain code with mock objects as parameters
  • Verify consistency in the mock objects

Citation
https://www.ibm.com/developerworks/library/j-mocktest/


From the blog Table of Code by Andon S and used with permission of the author. All other rights reserved by the author.

Wireframe Testing

A lot of software testing focuses on testing the code, which is very important, but there is another area of testing to consider, the user experience. When it comes to designing a product it is important to keep in mind who will be using it and what they are expecting. This area of testing ranges depending on the nature of the software. If, for example, you are developing software that is to be server based and will be run from command line it will require less time than a program that is to be used by the average employee.

It is a good idea to start this testing before you have invested too much time in developing the final user interface. A quote by Frank Lloyd Wright that summarizes this is “You can use an eraser on the drafting table or a sledge hammer on the job site.”

One way to accomplish this is to make use of wireframe testing. Wireframe testing is a method of testing that involves generating a mock up so that the end user can get a feel for what the final program will look like and how it will behave. There is also a variation where an expert analyzes the software instead of the end user.

Wireframes can vary in details. The simplest form may be a simple sketch the allows the user to see how items are laid out. More detailed wireframes fall into to categories high fidelity and low fidelity. Low fidelity wireframes are mock ups that provide a fairly similar look to the final product and some functionality where as high fidelity wireframes are very similar to the final product in look and feel.

Original Articles:
http://curioustester.blogspot.com/2015/08/wireframes-testing-part-i.html
http://curioustester.blogspot.com/2015/09/wireframes-testing-part-ii-how-did-i-do.html

From the blog CS@WSU – :(){ :|: & };: by rmurphy12blog and used with permission of the author. All other rights reserved by the author.