Category Archives: testing

An Intro to TestProject as an Automated Testing Tool

Classes restarted last week, and during the first week I found myself reading and writing a lot of introductory blog posts about upcoming semester course topics. As part of my setup tasks for CS-443 Software QA and Testing, I read about one of the most popular manual API Testing tools Postman and saw reference to TestProject, an automated testing tool from the same company Tricentis. So, I decided to do some further reading into TestProject as I understood the basics of manual testing, but had not seen automated testing in action yet. To do so, I found a blog post on the Tricentis site containing an overview tutorial of API testing and how platforms like Postman and TestProject can be valuable.

The post begins by discussing the value in API Testing and mentions the utility of Postman, but also brings up some of the limitations that are commonly encountered such as test automation, scheduling, and end-to-end test reporting. Intro TestProject → This testing platform offers solutions for end-to-end API testing by providing an environment to not only test API’s, manually but also to automate API-based test flows, schedule and run them periodically, and generate execution reports without the need for third party tools or writing any code.

The blog also contains a six chapter tutorial on using TestProject, so I delved into Chapter 1 – Basic API Test Automation. This emphasizes the platform’s ability to handle a wide variety of input sources like HTML, Databases, and more, then showing the application GUI and taking the reader through how to add ‘test steps’ in TestProject to set up automated testing for a GET HTTP request using NASA’s public APIs including search parameters and discussion on Dynamic Endpoint URLs (and implementing them) amongst others. Chapter 2 offers another brief tutorial discussing the value of scheduling automated tests with image steps (guide) to doing so with interactions to Android and iOS systems.

From these introductory chapters, I was able to get a basic idea of how to use TestProject to design calls, execute tests and access result reports. The other chapters in this TestProject tutorial cover more advanced API testing and validation flows, shell commands, scheduling API automation and more. As an introduction to Quality Assurance Testing and the course in general, this chapter was intriguing and valuable to get an idea of what an automated software testing tool looks like and how to use it in a basic sense. Stay tuned to read more about these other chapters and other topics in software quality assurance and testing and other exciting computer science related topics!

Sources:
General TP Post: https://blog.testproject.io/2020/11/10/automating-end-to-end-api-testing-flows/

Chapter 1: https://blog.testproject.io/2020/11/10/basic-api-test-automation/Chapter 2: https://blog.testproject.io/2020/11/10/api-test-automation-flows-combined-with-mobile-functional-test/

From the blog CS@Worcester – Tech. Worth Talking About by jelbirt and used with permission of the author. All other rights reserved by the author.

QAn’t Wait to See You Again

CS SERIES (16)Before I dive into my final installment of this CS series (for now), I wanted to say thank you if you actually read any of these posts and thank you to my new followers for dealing with the notifications you must have gotten.

For this last post, I wanted to discuss the article 5 Reasons You Are Wasting Your Testing Time by Joel Montvelisky. The author expresses how everyone can do testing but not everyone can do “good testing,” which incorporates more than just what is basic checking.

A short list of main reasons why a lot of people tend to waste time that could have been spent testing other things includes:

  • Not having clear goals
  • Not understanding what or how much the feature means to your End User
  • Not keeping track of what was tested and potential discoveries
  • Not consulting what you already know or using references
  • Not giving feedback that could be shared to help others

Sam CS (22)I agree with most of what Montvelisky says as I have personally noticed what has happened from my experience as a Software Quality Assurance Intern when any of the five things above took place based on a task. It is important to understand that sometimes these things may be out of your control but you should still try your best to avoid miscommunication wherever possible.

Montvelisky’s content has not necessarily changed the way I will work as I have already been consciously making an effort to understand what I am reviewing, how stuff is meant to work, logging tasks, connecting tasks to previous occurrences, and communicating with the QA team. If I had read this article before I started working, it would have been more useful as it would serve as a foundation to how one should think of testing beyond the basic functionalities.

Something I wanted to emphasize is the post-test reviews and feedback sessions with peers. I found that if someone else needed to learn about a project or a task had to be communicated with a client, logging any kinds of notes or information was better than not having anything prepared to discuss. They do not necessarily have to be posted for everyone to access but it would be good to note it in any sort of text editor for future reference. I think an example of when to note things is if you found something that does not prevent your current task from being approved but it still affects something for the overall program.

Overall, when testing things I believe you should trust your instincts on reporting things you find. It helps when you try to imagine how much it could affect a program in the long run if not brought to someone’s attention, even if it may seem minimal.

Best of Luck,

Sam

Article: https://qablog.practitest.com/5-reasons-you-are-wasting-your-testing-time/

From the blog CS@Worcester by samanthatran and used with permission of the author. All other rights reserved by the author.

Test Yourself Before You Wreck Yourself

CS SERIES (14)Testing, testing. I may need your approval on this article I read by Software Testing Magazine on Approval Testing. Approval testing, as defined by this article, is a way of software testing that results in presenting the before and after of an application for a user (ex: software development team) to review it and potentially approve it. It’s more of a visual representation of testing and one of the major cons is how the results have to be checked manually.

Some testing tools mentioned include: Approval Tests, TextTest, Jest, Recheck, Automated Screenshot Diff, Depicted (dpxdt), and etc.

The main purpose of the software testing tool, using TextTest for example, is checking that the text output after running program from the command line in different ways.

What I found interesting is how a user can see that a test technically could have “passed” or “failed” but still decide to mark it as the other because they choose what feature they are looking for in the end. This makes it a little more flexible to use approval testing as it is more of a guide or guideline for a user instead of only seeing one word and then a short description of what could have gone wrong. I think this process is much more transparent or descriptive with a user about what could have gone wrong or what went right.

One way the content has changed how I will think about the testing is how there are so many more types of software or programs out there than we can imagine which help us better code or create our own software and programs. This one is especially good for visual coders and testers who like to see their results firsthand to compare what they are expecting with what they actually got.

Overall, I found this article was useful because it introduced me to thinking about a better way of logging the differences between what the reference result is versus the actual result. I did not disagree with any of it since it showed us how we can use approval testing to our advantage while still being honest about its limitations.

Article: http://www.softwaretestingmagazine.com/knowledge/approval-testing/

From the blog CS@Worcester by samanthatran and used with permission of the author. All other rights reserved by the author.

Is Your Interface Two-Faced?

CS SERIES (11).pngWhen coding, users always have to be conscious about the way their code may be implemented or used in the future by different services leading to potential misuse. After reading Code Health: Make Interfaces Hard to Misuse by Marek Kiszkis, it made me think about how important communication between testing code is.

I found this content useful because Kiszkis featured some examples that can show how an interface could be misused easily. The examples included:

  • Requiring callers to call an initialization function
  • Requiring callers to perform custom cleanup
  • Allowing code paths that create objects without required parameters
  • Allowing parameters for which only some values are valid, especially if it is possible to use a more appropriate type

Sam CS (17)It is also good to remember that at the end of the day, code should be defensive but not too defensive to the point that complexity is increased and performance overall is reduced. Kiszkis says “it is not always practical to have a foolproof interface” because there will be situations where some requirements are things that cannot be expressed in an interface.

After seeing so much content based on being careful about what you code it is surprising how this article says it is not necessary to plan too hard. I kind of understand why Kiszkis would say this but personally, I kind of disagree with this. The reason why I disagree is because if someone does happen to end up with more time to work on something than expected and they know it will make something more efficient, then why not go for it?

Overall, I appreciated what was shared in this article in terms of encouraging users to try and see issues that can arise with their code when it comes to interfaces. The main takeaway for me is that if something is brought up, or triggered by undefined behavior, a user should try and make it impossible for this to happen. A way of doing so is by adding things where necessary, like certain slots in his example. It does not have to be too specific but detailed enough so that it covers different aspects, similar to how we try to prepare for everything when it comes to equivalence class testing.

Article: https://testing.googleblog.com/2018/07/code-health-make-interfaces-hard-to.html

 

From the blog CS@Worcester by samanthatran and used with permission of the author. All other rights reserved by the author.

The Business Case for Unit Testing

In this post, Erik Dietrich talks about the importance of making sure developers are well-trained in writing units tests.

“If you have developers writing good unit tests, you’ll develop a nice, robust unit test suite over the course of time. Developers will run this test suite constantly, and they’ll even have the team build to run it in a production-like environment.”

Making sure your units tests are well written and easy for maintainers to understand helps prevent regression, like a developer editing code and breaking functionality elsewhere.

“Testable code is better code, as measured by how easy it is to change that code.”

Writing better unit tests allows for bugs to be caught earlier and increases functionality and modularity, saving businesses money and making you a better developer with a more productive team.

The post The Business Case for Unit Testing appeared first on code friendly.

From the blog CS@Worcester – code friendly by erik and used with permission of the author. All other rights reserved by the author.

The Clean Coder: Chapter 7 & 8

This week in the Clean Coder I read chapters 7 and 8. These chapters covered a lot about testing. Acceptance testing and testing strategies to be specific. One of the more interesting topics that was talked about was estimates. This is an interesting topic to me because a few months back I listened to an interesting podcast on the same exact topic. Now Uncle Bob didn’t go into as much details here as did Steve McConnell on the podcast but he made the most important point:

Professional developers understand that estimates can, and should, be made
based on low precision requirements, and recognize that those estimates are
estimates.

The later-half of that quote is the important part. The statement that estimates are estimates is important. A lot of the time estimates are taken as absolute fact in industry and unfortunately this has become poor practice. Once we remember that estimates are estimates then we start taking uncertainty back into account and everyone is happier for it.

In chapter 8 Uncle Bob began talking about testing strategies. The first point he decided to hit was actually a reiteration of something he said in an earlier chapter, “QA Should Find Nothing”. My understanding initially is that as a developer you should make sure QA has NO role. However, this is not the case. The issue here was my view on the role of the QA engineers. I assumed that their job was to catch the bugs I missed. This is wrong. According to Uncle Bob their role should consist of creating automated acceptance tests and characterizing the true  behavior of an application.

The rest of chapter 8 continues to talk about the different types or stages of automated testing. These are things like; unit testing, component testing, integration testing, system testing and exploratory testing. These are all things I’ve talked about in previous blog posts so I won’t spend too much time talking about them. I do however, want to note one thing Uncle Bob mentioned,

Unit tests provide as close to 100% coverage as is practical. Generally this
number should be somewhere in the 90s. And it should be true coverage as
opposed to false tests that execute code without asserting its behavior.

It’s interesting that when he talks about code coverage he makes it a point to say that our tests should assert something about >90% of the code we’ve written.

That’s all for this week. I look forward to the next week’s chapters which talk about Management and go into more depth about Estimations!!

From the blog CS@Worcester – Tyler Lundstrom by CS@Worcester – Tyler Lundstrom and used with permission of the author. All other rights reserved by the author.

What is Reasonable Test Coverage?

Earlier this week I was reading through some blogs and stumbled across this particular piece that I found highly amusing but answered a question that I had been looking for an answer.

I found this article called The way of Testivus. It’s essential Confucius meets programming. While reading through and chuckling at most of the things mentioned on this page I stopped being side tracked and continued to see if I could find a good answer to my question – What is a reasonable amount of test coverage? As I continued to google I landed back onto the artima.com forums and found there was the question I had been searching for. Funny enough the writer of the “The way of Testivus” replied with an answer to this poster’s questions

Testivus On Test Coverage
Early one morning, a programmer asked the great master:

“I am ready to write some unit tests. What code coverage should I aim for?”
The great master replied:

“Don’t worry about coverage, just write some good tests.”
The programmer smiled, bowed, and left.

…

Later that day, a second programmer asked the same question.

The great master pointed at a pot of boiling water and said:

“How many grains of rice should put in that pot?”
The programmer, looking puzzled, replied:

“How can I possibly tell you? It depends on how many people you need to feed, how hungry they are, what other food you are serving, how much rice you have available, and so on.”
“Exactly,” said the great master.

The second programmer smiled, bowed, and left.

…

Toward the end of the day, a third programmer came and asked the same question about code coverage.

“Eighty percent and no less!” Replied the master in a stern voice, pounding his fist on the table.
The third programmer smiled, bowed, and left.

…

After this last reply, a young apprentice approached the great master:

“Great master, today I overheard you answer the same question about code coverage with three different answers. Why?”
The great master stood up from his chair:

“Come get some fresh tea with me and let’s talk about it.”
After they filled their cups with smoking hot green tea, the great master began to answer:

“The first programmer is new and just getting started with testing. Right now he has a lot of code and no tests. He has a long way to go; focusing on code coverage at this time would be depressing and quite useless. He’s better off just getting used to writing and running some tests. He can worry about coverage later.”

“The second programmer, on the other hand, is quite experience both at programming and testing. When I replied by asking her how many grains of rice I should put in a pot, I helped her realize that the amount of testing necessary depends on a number of factors, and she knows those factors better than I do – it’s her code after all. There is no single, simple, answer, and she’s smart enough to handle the truth and work with that.”
“I see,” said the young apprentice, “but if there is no single simple answer, then why did you answer the third programmer ‘Eighty percent and no less’?”

The great master laughed so hard and loud that his belly, evidence that he drank more than just green tea, flopped up and down.

“The third programmer wants only simple answers – even when there are no simple answers … and then does not follow them anyway.”
The young apprentice and the grizzled great master finished drinking their tea in contemplative silence.

 

In my early stage as a programmer I decided I am going to dedicate myself to not best understanding what I should be looking for in code coverage but better understanding how I can get there with quality tests.

From the blog CS@Worcester – Tyler Lundstrom by CS@Worcester – Tyler Lundstrom and used with permission of the author. All other rights reserved by the author.

Creating Tests with Laravel

Laravel, Artisan and Testing

By: Tyler Lundstrom

The past few weeks I have been working on a project using the Laravel PHP framework. So far I love it and it has increased my abilities and knowledge for developing PHP immensely. One of the many great things about Laravel is it’s Artisan tool. Laravel’s Artisan tool is used to to create a lot of the boiler plate code for you as well as put the generated files in the proper repositories.

The command I’d  like to talk about today is php artisan make:test this command when given a name will auto generate the test boiler plate you need to running unit tests in Laravel. This saves you time from having to manual type out all the files you are using as well as the class name and extensions. As a side note about the extension, you extend a class called TestCase which is an abstract class that has many of the boiler plate style syntax already filled out and you simply just extend upon that.

Once your file is created and you are ready to start creating your tests writing PHPunit is now as simple as creating jUnit code.

public function testExample(){ $this->assertTrue(true); }

That’s all you need to know about testing with Laravel. Thank you Taylor Otwell for making PHP development a delight!

From the blog CS@Worcester – Tyler Lundstrom by CS@Worcester – Tyler Lundstrom and used with permission of the author. All other rights reserved by the author.

Ugh! PHP Unit Testing

I tend to spend a lot of time programming in PHP in my free time. One of the things I have been looking at, since I am taking a software testing class, is PHPUnit and unit testing in PHP. I was getting discouraged with how odd PHPUnit seemed in comparison to the ease of use of something like JUnit. I stumbled upon this blog post that is a conversation between two people about PHPUnit.

The conversation starts with Ed explaining how he doesn’t like PHPUnit specifically:

Ed: I guess realistically my complaints are aimed at PHPUnit . It’s very powerful and very complete from what I can tell, but I think it’s difficult to pick up and I think that difficulty makes people less likely to use it. Because it’s by far the best known testing tool, I think that tends to limit the use of unit testing, period, in PHP.

He goes on to state how other languages have unit testing support that are far easier to use. Java and Python for example have extremely easy to use unit testing frameworks/built in features.

After a little while he continues to explain the reasons he believes PHPUnit is so intimidating:

Ed: I think boilerplate is part of the issue. I think that’s intimidating. Tools can mitigate that to some extent, but I don’t think it eliminates the problem entirely. I just don’t think writing a simple test should be anything more than a couple lines of code.

Previous to that quote the other host, Chris, gave a snippet of a simple unit test and it required 17 lines of code to simply write one assertion. That seems absurd!

At the end of the post Ed notes something that is very important and relates closely to TDD:

To write testable code, you really have to be thinking about testing when you write your code. It takes a bit of time to get used to that, but I think it’s very doable.

The number one reason that PHP developers have a tough time thinking in this manner is due to the fact that most of them are self taught. PHP has a really small learning curve which is great for people who want to get into web development, but in order for PHP unit testing to move forward and to better things, more developers have to get into the TDD mindset.

 

P.S. I find it amusing that in my WordPress (Mainly written in PHP) post editor, the acronym PHP isn’t included in the base dictionary.

From the blog CS@Worcester – Tyler Lundstrom by CS@Worcester – Tyler Lundstrom and used with permission of the author. All other rights reserved by the author.

Lessons to Learn from Kent Beck the father of TDD

In a podcast I listened to earlier this week from PythonTesting, Test & Code podcast, I was able to hear from the “father” of TDD himself Kent Beck. In this podcast, the host Brian took snippets from a Software Engineering radio podcast (Episode 167) that, what he thought, would have the most impact on the listener.

These are the 5 things Brian decided to look at:

  1. You’re tests should tell a story.

  2. Be careful of DRY, inheritance, and other software development practices that might get in the way of keeping your tests easy to understand.

  3. All test should help differentiate good programs from bad programs and not be redundant.

  4. Test at multiple levels and multiple scales where it makes sense.

  5. Differentiating between TDD, BDD, ATDD, etc. isn’t as important as testing your software to learn about it. Who cares what you call it.

One interesting thing I noted from this was point number 2. Being careful of writing test that align with common software design practices. This kind of went hand-in-hand with point 1 where he says your tests should tell a story. Each individual test should tell the person reading it, what was being accomplished at that given point. For example if you try to use the Don’t Repeat Yourself (DRY) philosophy then you won’t be able to see that story as well in your tests because you’ve tried consolidating the repeats.

From the blog CS@Worcester – Tyler Lundstrom by CS@Worcester – Tyler Lundstrom and used with permission of the author. All other rights reserved by the author.