Category Archives: cs-wsu

11 Things You’re Doing Wrong When Testing Code

Full Article by Matt Lacey

In this post, Matt Lacey discusses 11 things that developers do wrong when creating and testing their code.

The first one he lists is that developers can just fall into the trap of just not making any tests for their code. He states that a way to fix this is to just start creating tests for the code that you’re already working on and plan to always do this for future projects.
The next item his list is that developers need to design and make tests when they start their projects. This makes it easier to implement into the code that is currently being worked on rather than adding in tests later. Also adding in tests later may require developers to make architecture changes and will add on additional time.
The third item on the list is writing failing tests. Lacey goes into explaining that your should not specifically make tests to fail (or have a red outcome) because the point of writing tests is to make sure that our system is working properly.
The fourth thing on the list is having a fear of unimplemented tests. Having these types of tests (which also have a proper names for the functions they are preforming) will serve as a reminder about the intended function of the system as well as what the test should accomplish.
Fifth on the list is simply not giving your tests good, identifying names. Lacey states that the way you name your tests isn’t important; what matters is that your naming behaviors are consistent, and properly describe what the tests are suppose to do.
The next item is making tests that do too much. Here he gives a reminder that your tests should only be testing one thing; this could even be a series of simple things that make up one task even, but it should not be testing multiple long tasks.
Seventh on Lacey’s list of what not to do is not testing you code. Lacey points out that creating mocks and running them does not actually mean you are testing the code. He states that these types of “tests” could just end up being a waste of time and effort.
The next thing on the list is worrying too much about code coverage. Lacey states the knowing “how much of the code is executed when the tests are run should be useful but because it doesn’t consider the quality of the tests that are executing the code it can be meaningless.” He goes on to say that code coverage is only of interest if the coverage is too high or too low. Lacey then address what you kind of code should be tested then by stating “Does the code contain non-trivial complexity? If it does then you need some tests. If it doesn’t then you don’t.”
The ninth item that Lacey views as a mistake is only doing one type of testing. He states that if you’re only making one type of tests then you are not properly testing your software. It takes a multiple different tests to find mistakes in all areas of your system.
Number ten on the list focusing too much on short term tests. Tests should not be made to test something that is static; like if the code is written correctly. Instead tests should be written the make sure that the code continues to function correctly over time. Tests should be able to be run repeatedly to make sure that no new errors occur if new code is added or changed.
The last item on Lacey’s list of what not to do is being a developer that relies on someone else to create and run tests. Tests should always be available for developers to run if the need arises. If there is something that will hinder a quick execution of tests then that will just prevent developers from running the tests they need. The best thing to do, according to Lacey, is to make sure that “code and tests…be kept in the same place and any setup needed should be scripted.”

From the blog CS WSU – Techni-Cat by clamberthutchinson and used with permission of the author. All other rights reserved by the author.

Testing IoT Devices: Key Areas

Full Article by Pavel Novik

In this article, Pavel Novik talks about the phenomena known as the Internet of Things (IoT), and goes over the importance for companies to properly tests the software that is used by these products.

Pavel Novik points out that many companies use IoT products, but more than half of them “do not have a software testing strategy for this specific aspect of their IT infrastructure.” However being able to test certain devices can be an issue, Novik points out, if the company only sells the application for running the devices because they might not be able to have all the products on hand to tests. Novik suggests that companies that are in this position to find out which devices their customer base buy the most, and then limit their testing to those products.

Novik then goes on to talk about what areas should be tested when it comes to IoT devices. The first one that he suggests is security; making sure that no one is able to steal personal information from the devices, and also prevent attacks from happening which may lead to accidents. The second area is performance; this is to make sure that all the data that is being sent between these products without fail even when some sort of disruption takes place. The third area Novik points out is functionality; this is testing “to verify that the application functions meet all the functional requirements,” and that testers should “strive to go beyond general test scenarios, and consider body movements, voice commands and sensor utilization when writing test cases.” The fourth and last area that’s cover is compatibility; covering this area while testing will make sure the product works with other versions, devices, and systems that it you may encounter further down the line.

From the blog CS WSU – Techni-Cat by clamberthutchinson and used with permission of the author. All other rights reserved by the author.

When to Mock?

Full Post by Uncle Bob

In this blog post, Uncle Bob goes over the basics when you should use mocks, and the advantages and disadvantages of using them.

He first goes over what will happen to your test suite if you have no mocks (he’s also going under the assumption that this is for a large application). The issues that arise when no mocks are used is that, during execution, your tests could take from several minutes to several hours to run. The next problem that could happen when no mocks are used is that your test suite might not cover everything. Bob states that “error conditions and exceptions are nearly impossible to test without mocks that can simulate those errors.” The example he gives for this is if you have files or databases tables that need to be deleted, this task my be to dangerous without the use of mocks. The last thing Uncle Bob brings up is that tests are really fragile if you don’t use mocks. This is because tests can easily be disrupted by faults in the system that are not even caused by what you’re testing.

This may seem that mocks are always the way to go, but Uncle Bob covers what happens when you use too many mocks (for this Uncle Bob goes under the assumption that you have mocks between all classes in your test suite). This first issue is that “some mocking systems depend strongly on reflection,” and if you have too many this can slow down your testing phase. The other thing is that, if you use mocks between all classes, you may end up making mocks that create other mocks. This can create two problems. One, the set-up code you’re using can end up getting convoluted. Two, this can lead your “mocking structure [to] become tightly coupled to implementation details causing many tests to break when those details are modified.” The last thing Bob points out is if you have too many mocks, you may end up have to create extra interfaces “whose sole purpose is to allow mocking,” and this can lead to “over-abstraction and the dreaded ‘design damage’.”

This may make it seem that using or not using mocks can lead to issues, so what is the correct why to use mocks in your test suite? Uncle Bob states that the method he uses for this is to only “mock across architecturally significant boundaries, but not within those boundaries”; this means to only mock when it comes to using any external software or anything that is rather large, like a database or web server. This way of mocking will address most of, if not all, the issues that came up when having no mocks or too many. The other thing that Uncle Bob suggests doing is to always create your own mocks and not rely on other software tools to do it. He points out that most mocking tools have their own language that can become time consuming to learn. Also creating your own mocks maybe more beneficial in the end because it forces you to name your mocks and put them in directories which allows you to use them for other tests. The other benefit to creating your own mocks is that it makes you take time to create your own mocking structure.

In the end, Bob suggests using mocking as little as possible and to strive to design tests to not use mocks.

From the blog CS WSU – Techni-Cat by clamberthutchinson and used with permission of the author. All other rights reserved by the author.

4 Keys to Better Test Data Management

Original Article by Jeffrey Hughes

Finding proper test data for testing the software you’re developing for can be a challenge most of the time for a variety of reasons. In his article, Jeffery Hughes goes over why test data management is important and lists four main ways of how to improve your management of test data.

Hughes states that test data management is important because it can help address challenges that occur in providing test data quickly and continuously for the application you’re working on. The main three challenges that he touches on are test data compliance, availability, and freshness.

After that, Hughes addresses what the four keys to better test data management are. The first key is how to properly discover and analyze test data. Hughes states that your test data may be spread across multiple platforms and could be extremely time consuming if you tried to access it all manually. He suggests that using a tool that can make sure that the right test data is being tested and to come up with a “solution that provides complex coverage analysis and data visualization is a better approach.”

The second key that Hughes talks about is protecting sensitive data. In this section, he points out that is important make sure that sensitive data is being properly secured (even when just in testing), and to make sure that you use “inconsistent data to test for anomalies that are sure to occur with end users.”

The third key is make sure that you’re able to deliver test data on demand. Hughes points out that it is a great advantage to have your data stored on a central repository so that all test data is in one place and can be easily shared. The hope is to make sure that there are no more bottlenecks for testers that are “waiting for data” when running tests.

Finally the fourth key is to make sure that you try to automate as many steps as you can. Hughes suggests severally tools that can make automating your testing easier for your and your team. He states that this kind of “stabilized automation framework guarantees that data will be provisioned in the right state when needed by your testers.” This also makes it easier to trace your data and test cases back to their requirements, and also means that if your “requirements change, the data can be updated automatically.”

From the blog CS WSU – Techni-Cat by clamberthutchinson and used with permission of the author. All other rights reserved by the author.

Google Testing Blog: Hackable Projects – Pillar 2: Debuggability

Hackable Projects – Pillar 2: Debuggability

This is a continuation of Patrik Höglund series about how to make your software hackable. (If you have not read the first article, here is the link: first article)

Patrik Höglund continues to talk about how to make your code more hackable by going on to his second pillar of hackability: Debuggability. However here this word doesn’t simple mean running a debugger on your code and calling it a day. Höglund explains debuggability as “being able to easily find what’s wrong with a piece of software, whether it’s through logs, statistics or debugger tools.” To help with this process, Höglund goes over a few examples that help their developers with debuggability.
The first one is running your project on a local host. This way you not only get around the possibility of crashing your main servers, but you can also set up “scripts that invoke the server binaries with flags.” This helps to make debugging and developing your server a simpler process overall. Höglund then goes into how to debug for mobile apps. He suggests using unit tests here as well since they have great hackability. He then gives some suggestions for software to use for debugging on iOS and Andriod. Höglund then some tips for when debugging gets tricky.
The next step that Höglund brings up is making sure that you have proper logging.He links another article to look at, but he states that you should make sure that you are getting proper stack traces instead of going off of an error message. He also says that there is such a thing as too much logging which can lead to having your project be less hackable. The other thing Höglund brings up to make sure that you can use statistics to your advantage. If you run a monitoring system, like Stackdriver for Google Cloud, Höglund says that “tools can help you keep track of request volumes and error rates.” This lets you find errors quicker and allows you to rollback bad code as soons as possible.
The last thing that Höglund talks about is System Under Test (SUT) Size. He says that “small and focused tests aid debuggability, because when a test breaks there isn’t an enormous SUT to look for errors in,” and makes tests more reliable. He refers to this article for information about SUT.

From the blog CS WSU – Techni-Cat by clamberthutchinson and used with permission of the author. All other rights reserved by the author.

Google Testing Blog: Hackable Projects – Pillar 1: Code Health

Hackable Projects – Pillar 1: Code Health

In this blog post, author Patrik Höglund talks about how over the years, software development can become stressful to deal with and fix constant issues. The way he suggests to resolve this issue is by making the software more “hackable”; not in the sense of making the software more vulnerable to attacks, but making it easier to modify from a developers stand point. Höglund goes on to say that a hackable project is one that includes easy development, fast builds, good and fast tests, clean code, easy running and debugging, and “one-click” rollbacks. Höglund then goes on to describe the three main pillars of hackability, which are code health, debuggability, and infrastructure.

This post focuses solely on the first pillar of hackability: Code Health.
The first thing Höglund covers are tests that you should use. He says that “unit and small integration tests are probably the best things you can do for hackability” rather than using end-to-end testing. The other thing to testing is that if you have poorly tested legacy code, the best thing to do is refactor it and add tests along the way. Even though this can become time consuming, it’s work it in the end because it leads to a more hackable system in the long run.
The next thing that should be done is to make sure that your code is readable and goes through code review. This means that there should be a reviewer who looks over the code changes to make sure that the new code is consistant with the rest of the code. The changes should also be small and coded cleanly so as to make it easy if a rollback is necessary. Another thing that will help with hackability is making sure that all of your code is submitted in a constistant format.
To reduce risks even more, you should try to consistantly have a single branch for your project. This not only decreases the possible risks, but also reduces the expense of having to run tests on multiple branches. This could possibly back fire though if, as Höglund writes, “Team A depends on a library from Team B and gets broken by Team B a lot.” Höglund suggests that “Team B might have to stabalize thier software for them to use this method and have hackable software.
The last things that Höglund focuses on for Code Heath is making sure that your code has loose coupling, testability, and ways that you can aggressively reduce technical debt.

From the blog CS WSU – Techni-Cat by clamberthutchinson and used with permission of the author. All other rights reserved by the author.

What Classes Should Be Tested

Testing on the Toilet: Prefer Testing Public APIs Over Implementation-Detail Classes

In this blog post by Andrew Trenk, Trenk goes over what code needs to be tested, and what doesn’t, when making a Public API. Trenk goes on to explain that if a simple piece of code has only one user or is used once in another class, you should create tests for the other class or not bother testing the simple code at all. He says that anything that is an implementation detail should not get it’s own test.
In fact if you do create tests for this implementation, it might make the code and tests harder to maintain over time, it could cause issues when trying to refactor, and it could also hide issues in your code that only show when using the API. Overall Trenk suggests testing your implementation details indirectly through your API classes. This way you see that they work with your API and you have fewer tests that will need to be updated.

From the blog CS WSU – Techni-Cat by clamberthutchinson and used with permission of the author. All other rights reserved by the author.

How Google Deals with “Flaky Tests”

Read the Full Article

This post, by John Micco, talks about how the Software Developers at Google deal with and minimize the damage of “Flaky Tests”. Micco defines a flaky test “as a test that exhibits both a passing and a failing result with the same code.” He goes on to state that this causes major problems because how difficult it is to find the cause of the flaky tests and the frequency of which these kinds of tests appear. There is also the issues of a flaky tests being dismissed only to find out later that it was a real failure which sometimes results in passing through broken code.

There are several strategies that the developers at Google use to minimize the damage and confusion that is caused by these flaky tests. One way is to only report a true failure if the test fails three times in a row. This, however, can be costly if the test that is being run takes a large amount of time to complete. Another way is that they implement a tool that monitors the flakiness of a test. If the tool determines that the flakiness for a test is too high, it removes the test from the critical path. The last strategy they have is to use another tool that monitors flakiness levels. When the levels change, the tool tries to find the reason for the test’s change in flakiness.

From the blog CS WSU – Techni-Cat by clamberthutchinson and used with permission of the author. All other rights reserved by the author.

The Pros of Platform-led Testing

Platform-led Testing by Lakshminarasimhan Rajabather

In this article, Rajabather goes over the positives of using platform-led testing to help with automation across all the stages of the software development life cycle. The first advantage that he lists for using this approach is that it cuts down on the cost and time to use assurance across the development’s life cycle. This is also positive to businesses since their goal is to minimize cost and time, and maximize quality.

Rajabather points out is that platform-led testing makes sure that the software is constantly being checked and validated at every stage of development. This is able to be done because, as previously stated, it is now possible to have software assurance constantly being implemented through the entire life cycle. Platform-led testing also works well with Agile development because it “promises quick sprints, rapid sign-offs and a measurable transfer of value from one sprint to the next.”

The last two advantages of platforms that Rajabather lists show that they are beneficial to businesses. He states that platforms allow businesses to build upon both industry and third party analytical tools which makes it so that the tools can be customized to fit a certain need. The last benefit is that platforms are not “restricted only to the requirements, design or execution stage of the lifecycle.”

Please read the full article for more information on this subject.

From the blog CS WSU – Techni-Cat by clamberthutchinson and used with permission of the author. All other rights reserved by the author.

Blog Introduction Post

This blog is for Worcester State University CS-443 and will be about upcoming news regarding technology and developments in the Computer Science field.

From the blog CS WSU – Techni-Cat by clamberthutchinson and used with permission of the author. All other rights reserved by the author.