Category Archives: Software Testing

Code Reviews for Quality Assurance

As we continue in CS443 – Software Quality Assurance and Testing, we’ve delved into the test strategy/skill of group code reviews for debugging. So, this week I chose to search for a blog about effective code reviews with a focus on good practices to extract the most value from participating in them and found a valuable article posted on Vates.com. This article is from August 2023, so it is modestly recent and applicable to our current learning materials and discusses best practices and habits.

Code review is a pivotal practice in software development, offering more than just bug detection. It serves as a collaborative process where developers meticulously inspect code changes before integration into the main codebase. By doing so, it not only identifies defects but also enhances code quality, ensures alignment with project requirements, and upholds coding standards. The process fosters collaboration among team members, facilitates knowledge sharing, and integrates multiple perspectives, ultimately leading to the creation of reliable, maintainable, and efficient software.

To maximize the benefits of code review, it’s essential to implement effective strategies. This begins with establishing clear objectives tailored to the project’s goals. By defining specific focuses, such as code quality improvements, security enhancements, or performance optimizations, teams can streamline the review process and prioritize areas of concern.

Moreover, involving the right team members is crucial. Senior developers, subject matter experts, and team leads contribute diverse viewpoints, enriching the review process and leading to better outcomes. Adhering to a comprehensive code style guide ensures consistency in coding practices, which is vital for long-term maintainability. Utilizing code review tools further enhances efficiency by automating checks, highlighting changes, and facilitating collaborative discussions.

Promoting a culture of constructive feedback is equally important. Emphasizing that reviews aim to enhance overall code quality rather than criticize individuals encourages open communication and continuous improvement. Balancing review speed with depth ensures timely feedback without compromising quality, fostering agility while maintaining code integrity.

Lastly, following up on review feedback is essential for iterative improvement. Addressing comments promptly, clarifying questions, and implementing necessary changes contribute to a learning culture within the team, leading to continuous enhancement of development practices.

Vates is a leading provider of IT and software services and the host of this blog. They provide tools and resources to enable teams to unlock the full potential of code review, IoT consulting, and software testing services. This collaboration empowers organizations to elevate their software development practices, delivering exceptional value to stakeholders through collaborative development and effective code review implementation.

Source:

https://www.vates.com/the-art-of-code-review-effective-strategies-for-collaborative-development

From the blog CS@Worcester – Tech. Worth Talking About by jelbirt and used with permission of the author. All other rights reserved by the author.

Decision Tables from a Template

Over the past few weeks in CS443 – Software Quality Assurance and Testing, we’ve been learning how to apply our boundary test classes to create Decision Tables and apply somewhat similar logic to create Program and DD-Path Graphs for code segments. Decision tables are visual tools used in software testing and analysis to specify actions based on given conditions. The strategy we learned in class of assessing all possibilities then systematically combining them based on the decision outcomes and particularly “Don’t care” scenarios seems like a useful and interesting way to map out test designs.

So, I decided to look into blogs discussing Decision Tables and their implementation in software testing and found a great post on ShiftAsia with abstract and specific examples alongside general discussion. This post is also quite recent – posted on January 9, 2024 – which is something I always appreciate as the software/tech world is constantly changing. It opens by describing how to create a Decision Table by representing it with the following matrix:

Condition Stub Condition Entries
Action Stub Action Entries

Condition stub: List of all conditions in consideration

Condition entries: Filled out with Y/N (or X) to cover all possible combinations of conditions

Action stub: List of all possible actions/output

Action Entries: Marked (generally with X or blank) to show outcome and an association between a condition and result.

This is then illustrated with an example of being able to register according to conditions of having a valid email, registered email, and valid password. I found this template and example helpful to better understand Decision Tables in general by comparing them to the steps we did in our In-Class Assignment 7. And, using the example of an altogether invalid email forcing all results to be “Invalid” makes sense logically for the column consolidation.

The process of combining columns and simplifying Decision Tables is reminiscent of CS254 – Computer Architecture and Organization concepts, particularly using K-Maps to calculate Sum of Products and Product of Sums. Based on similar responses to a variety of inputs, we are able to essentially combine and simplify the K-Map table and in turn the expression it produces. While K-Map logic works based on binary math laws rather than actual outcomes, there’s a clear correlation here as we represent outcomes with boolean values that can be easily represented in binary – as either a 0(false) or 1(true). My personal experience in CS254 wasn’t the best – I didn’t totally understand how many of the concept we learned are applicable in practical situations, so it’s cool and exciting to see it applied in software testing – an area I would’ve probably least expected it.

Sources:

https://blog.shiftasia.com/use-decision-table-in-software-development

From the blog CS@Worcester – Tech. Worth Talking About by jelbirt and used with permission of the author. All other rights reserved by the author.

Another Look at Boundary Value Analysis and Equivalence Class Partitioning

Recently in CS443 – Software Quality Assurance and Testing we’ve been learning some of the conceptual aspects of code testing that are required to identify the relevant points of programs to test as likely break points. We’ve primarily learned about Boundary Value and Equivalence Class testing strategies, so I decided to find a blog to learn more about each of these from a third-party perspective. I landed upon a (relatively) recent blog on TestSigma – a (automatic) testing platform – from June 2023.

The post discusses the overall importance of software testing in ensuring functionality and reliability of software products focusing on the defining aspects of the two methods we’ve been learning: ‘Bound. Value Analysis’ (BVA) and ‘Equiv. Class Partitioning’ (ECP). BVA concentrates on testing the boundaries of a system to identify vulnerabilities, while ECP groups similar items into equivalence classes, helping testers target specific areas with a higher likelihood of containing bugs.

Benefits of applying BVA and ECP in software testing include improved understanding of the system, simplified test design, better test coverage, prioritization, and risk management. The applications of these techniques extend to various scenarios, such as database testing, network testing, hardware testing, time-based functionality, and UI testing. An interesting point that the article emphasizes is that BVA and ECP are often used together, providing an example of testing a form that accepts age as a number. It suggests partitioning the age range into groups for more effective testing while also considering likely break points.

Common challenges discussed to avoid when using BVA and ECP include restricting testing to input values alone, making assumptions about limits and classes, ignoring user behavior, over-relying on these techniques, and neglecting edge cases. The post concludes by comparing BVA and ECP, highlighting their differences in testing approaches and summarizing them as thought processes that enhance testers’ understanding of the system, leading to improved test coverage and strategy.

Test automation for BVA and ECP using tools like TestSigma (or other softwares) is also discussed, highlighting the potential benefits of saving time, ensuring accuracy, and achieving better test coverage. However, the decision to automate tests should be made considering the cost and benefit of automation and set-up.

Overall, this post taught me some interesting differences between BVA and ECP as well as reinforcing the benefits and basics we learned in class. One interesting aspect of this blog that I noticed in review is that it was written by author Apoorva Ram, a non-white woman in the computer science and specifically software engineering industry. This demographic represents a sparse minority in the computer science field and worth recognizing alongside their contribution with this and other blogs.

Sources:

https://testsigma.com/blog/boundary-value-analysis-and-equivalence-class-partitioning

From the blog CS@Worcester – Tech. Worth Talking About by jelbirt and used with permission of the author. All other rights reserved by the author.

Interesting Features of JUnit 5

Since beginning to work with code in CS443 – Software Quality Assurance and Testing, we’ve used JUnit framework for designing and running our test cases. So I decided to search for a blog post discussing some interesting features that I may not have come across yet, but could be useful and landed upon Exploring the Exciting New Features of JUnit 5. This post is from December 2023, so it should be relatively up-to-date and I recall a conversation with Dr. Wurst at one point where he briefly mentioned considering switching to a newer version of JUnit for some attractive features – hopefully we can delve into some of these.

Several feature additions come with JUnit 5 and specifically version 5.4. One that immediately stood out to me was support for more/new annotations and assertions like @Nested. We’ve looked at some basic annotations like @BeforeEach and @AfterAll in class but the idea of nesting tests is newer – however it makes perfect sense from a practical perspective. Depending on the outcome of an initial test, testers may want to run further tests on one branch or two different tests depending on which branch is followed. Proper annotating likely helps the compiler recognize the nested nature of the tests preceded and manage potentially complex webs of nested tests most efficiently.

There’s also improvements to the assertEquals() functions and overall flexibility through enhancements to API and insertion features for handling Lambda functions. This goes hand in hand with a new feature of JUnit 5 – the ability for tests to be dynamically generated during test runtime and implemented (if needed) using a Factory class/method. Last semester in Software Construction Design and Architecture, we learned about the Factory architecture and methodology so it was cool to see it applied to enhance features in professional software. 

Another cool feature of JUnit 5 which represents a considerable change from JUnit 4 is the transition to a modular structure, meaning there is a separate test runner and classes which operate independently from the main program. I could imagine that this separation isolates any issues that may arise during testing and protect the main program, while also preventing any unintended interactions with the main from interacting with properly designed tests.

JUnit 5 offers some major features and enhancements over the previous versions with the ability to tag and implement nested tests, improved Lambda function support and Factory method for dynamic test creation and implementation. Considering these, I can see how JUnit could be effective for designing automated test runs. I’m looking forward to implementing more of these features in our class and homework activities for CS443, and trying some extra tests and methods that I read about in this.

Source: 

https://blog.machinet.net/post/exploring-the-exciting-new-features-of-j-unit-5

From the blog CS@Worcester – Tech. Worth Talking About by jelbirt and used with permission of the author. All other rights reserved by the author.

TestProject Tutorial Conclusion – Advanced API Testing and Scheduling

We return again this week looking at the online TestProject blog tutorial focusing on the final two chapters 5 and 6 focusing on Advanced API Testing Automation and Scheduling API Automation Flows and CI/CD Execution respectively. In class, we’ve been working with JUnit integrated with VSCode so it’s been interesting seeing a very different User Interface in TestProject.

Chapter 5 looks at “Advanced” API Testing Automation, which primarily looks at more complex interactions and tests involving JSON objects and schemas. It also goes into some other more complicated calls and tests such as formatting URL-encoded requests, reading and using predefined user data sets and tests involving dynamic parameters. This chapter references public NASA API and tools; a key component that stuck out to me was the error report file generation that’s shown which easily identifies and organizes issues. Compared to previous chapters, I found this one to be less relatable and applicable to the things we’re doing in class, but I still learned a lot and was intrigued by the methodologies for test situations with multiple JSON paths to one target.

Chapter 6 focuses on the scheduling aspect of testing automation in TestProject and interactions with CI/CD/CMD pipelines. As always, there are an abundance of screenshots and images to walk readers through an example test set-up – beginning with the interface for scheduling tests and TestProject’s system of creating and assigning ‘jobs’. Tests are aggregated into jobs (typically as a bundle) which can then be executed as a one-time event or assigned a recurrence time frame. The interface to do so is clear and intuitive and reminds me a lot of CS383 – Cloud Computing where we are working on modules in AWS Academy learning about Amazon Web Services. AWS uses a similar interface and logical structure for assigning roles, jobs, permissions and many other facets making it intuitive for me to follow this TestProject tutorial. This chapter also discusses testing within a Docker container, which we used to implement for Dr. Wurst’s assignments but have recently switched to GitPod.

With this reading, we conclude the TestProject tutorial I originally found at the beginning of this semester. There’s been a lot of really valuable material and examples within these tutorials, particularly in chapters 1-4 as they focus on beginner concepts and I’ve just been getting started with learning about software testing and quality assurance. Probably most interesting and encouraging from this set of tutorials was how frequently concepts came up from other courses like Database Design and Cloud Computing. Software that interacts with those areas must be tested too so it’s important to know how to work my way around them and see visual examples of tests being designed and executed. In conclusion – TestProject seems like a great platform with many features, particularly an intuitive scheduling component, however JUnit’s interface remains my current favorite.

Sources:

Tutorial Intro: https://blog.testproject.io/2020/11/10/automating-end-to-end-api-testing-flows/

Chapter 5: https://blog.testproject.io/2020/11/10/advanced-api-test-automation-and-validation-flows/

Chapter 6: https://blog.testproject.io/2020/11/10/scheduling-api-automation-flows-and-ci-cd-execution/

From the blog CS@Worcester – Tech. Worth Talking About by jelbirt and used with permission of the author. All other rights reserved by the author.

TestProject – API Test Automation with Databases and Shell/File Interactions

This week, I continued reading through the TestProject Tutorial blog I began last week moving onto Chapters 3 and 4, which focused on API Test Automation involving database communications, shell commands, and local file interactions.

The Chapter 3 tutorial focusing on database implementation provides examples of testing connections and querying common types of databases, including Oracle Database, Teradata Database, and MySQL Database using TestProject’s Database Addons. It walks through a specific example of testing a connection to an Oracle Database, showing the successful establishment of the connection. This is taken a step further with an example showing the steps of creating a new table with data extracted from an API request, then setting up and performing an HTTP GET request extracting JSON paths and data from the API response and creating a dynamic query into an Oracle Database Table. I found this portion to be particularly valuable as these tasks seem highly applicable and like common requests of a software dev./analyst, like I plan to be after graduating in May.

Chapter 4 focuses on API test automation flows that incorporate a combination of Shell Commands and/or Local File interactions. As an example demonstration, it shows the steps to access a web page and extract the text using CMD shell commands in conjunction with a GET request, then writing the web page contents to the text file. These functionalities can also be easily implemented using SSH commands, which is also demonstrated. Finally, it goes into how to create and edit/interact with .csv files as a part of the test automation workflow, which can be used for test results and report generation (amongst others).

After going through four of the six chapters in this TestProject tutorial, I can confidently say that it’s a valuable resource and reference in learning about software testing and specifically automation, even if I never end up using the platform. I have been finding the examples to be extremely helpful in demonstrating how to actually use the software and perform some simple real-world tasks, breaking it down into individual test steps. While CS courses commonly include some form of tutorial or demonstration exercise, I personally find that they are often simplified to such a degree that the examples are “unusable”, in that no professional in a real situation would encounter and deal with such a simple task. However, these examples go into more complex applications, such as the earlier example using a combination of commands to extract text from a website, create a new file and write to it. Also, the examples within these tutorials tend to cover a variety of platforms/methods depending on the topic (i.e. a variety of popular databases) maximizing the likelihood for the content to be useful to readers. I’m looking forward to finishing out the last two chapters soon!

Sources:
Tutorial Intro: https://blog.testproject.io/2020/11/10/automating-end-to-end-api-testing-flows/
Chapter 3: https://blog.testproject.io/2020/11/10/api-test-automation-flows-combined-with-database-flows/
Chapter 4: https://blog.testproject.io/2020/11/10/api-test-automation-with-combination-of-shell-commands-and-local-files/

From the blog CS@Worcester – Tech. Worth Talking About by jelbirt and used with permission of the author. All other rights reserved by the author.

An Intro to TestProject as an Automated Testing Tool

Classes restarted last week, and during the first week I found myself reading and writing a lot of introductory blog posts about upcoming semester course topics. As part of my setup tasks for CS-443 Software QA and Testing, I read about one of the most popular manual API Testing tools Postman and saw reference to TestProject, an automated testing tool from the same company Tricentis. So, I decided to do some further reading into TestProject as I understood the basics of manual testing, but had not seen automated testing in action yet. To do so, I found a blog post on the Tricentis site containing an overview tutorial of API testing and how platforms like Postman and TestProject can be valuable.

The post begins by discussing the value in API Testing and mentions the utility of Postman, but also brings up some of the limitations that are commonly encountered such as test automation, scheduling, and end-to-end test reporting. Intro TestProject → This testing platform offers solutions for end-to-end API testing by providing an environment to not only test API’s, manually but also to automate API-based test flows, schedule and run them periodically, and generate execution reports without the need for third party tools or writing any code.

The blog also contains a six chapter tutorial on using TestProject, so I delved into Chapter 1 – Basic API Test Automation. This emphasizes the platform’s ability to handle a wide variety of input sources like HTML, Databases, and more, then showing the application GUI and taking the reader through how to add ‘test steps’ in TestProject to set up automated testing for a GET HTTP request using NASA’s public APIs including search parameters and discussion on Dynamic Endpoint URLs (and implementing them) amongst others. Chapter 2 offers another brief tutorial discussing the value of scheduling automated tests with image steps (guide) to doing so with interactions to Android and iOS systems.

From these introductory chapters, I was able to get a basic idea of how to use TestProject to design calls, execute tests and access result reports. The other chapters in this TestProject tutorial cover more advanced API testing and validation flows, shell commands, scheduling API automation and more. As an introduction to Quality Assurance Testing and the course in general, this chapter was intriguing and valuable to get an idea of what an automated software testing tool looks like and how to use it in a basic sense. Stay tuned to read more about these other chapters and other topics in software quality assurance and testing and other exciting computer science related topics!

Sources:
General TP Post: https://blog.testproject.io/2020/11/10/automating-end-to-end-api-testing-flows/

Chapter 1: https://blog.testproject.io/2020/11/10/basic-api-test-automation/Chapter 2: https://blog.testproject.io/2020/11/10/api-test-automation-flows-combined-with-mobile-functional-test/

From the blog CS@Worcester – Tech. Worth Talking About by jelbirt and used with permission of the author. All other rights reserved by the author.

CS443 – Introduction

Hello everyone, we are back and doing it again this semester. Tech Worth Talking About returns with posts about two courses, CS443 – Software Quality Assurance and Testing as well as CS448 – Software Development Capstone.
CS443 will focus more on strategies and tools for automating code testing as well as how testing incorporates into the overall development process. This goes far beyond simply debugging.

Welcome, and stay tuned!

From the blog CS@Worcester – Tech. Worth Talking About by jelbirt and used with permission of the author. All other rights reserved by the author.

If It’s Broke Don’t Fix it: Treating Bugs Like Buddha

For my last blog for this course instead of taking one
particular course subject, summarizing it, and theorizing about what actual
implementations may look like – I’d like to look at the whole course and do the
same. More specifically, I would like to cover how bugs or defects are actually
addressed, or not, in Software Development and Testing. To do this I have found
some interesting blog posts which argue that 100% of bugs may be able to be
fixed, but shouldn’t be. Instead, one should focus on serving the vast majority
of users under expected circumstances.

           Both blogs
focus on how impractical, and expensive, maintaining 100% stability or up-time
is. In the first focusing on bugs specifically they give many reasons why this
goal is unadvisable. The first is the prevalence of Agile Development,
dominating nearly the entire software development landscape. As such, the constraints
of this fast-paced development style limit the ability to do traditional QA testing;
if a program can have several revisions in a week, maybe even a day, then how
could a team reasonable test all these iterations. Instead, the author suggest
a stability monitoring tool to automatically test each revision.

           In
addition, they suggest that eliminating 100% of bugs would eliminate many which
users would never see, so why waste resources addressing them? Even if you
could fix everything how could you possibly know? This focus is reinforced when
considering the truly incredible breadth of devices that one may have consider,
and specifically in mobile development: where they cite the over 24,000
different android devices on the market. One must focus on the average expected
user experience and not waste time fussing with the outliers until, presumably,
a bug report is filed.

The second blog discusses
defects in systems in a similar way, covering more or less the same points, although
mentioning a rather obscure possibility: being legally challenged for claiming
that your product has no defects. Instead, what I believe they are trying to emphasize
is that products are always fallible, and the amount of resources required to
get them even close is impossible or impractical. As a result, as we move
forward in this class and towards graduation I think we should resist the impulse
to try for complete perfection and instead focus on what is achievable and
provides the best experience for the majority of users.

Sources

Not all bugs are worth fixing and that’s okay
The Zero Defect Fallacy

From the blog CS@Worcester – Press Here for Worms by wurmpress and used with permission of the author. All other rights reserved by the author.

Clarifying Edge Testing

I happened upon a very
interesting blog recently, searching for more examples of edge case testing,
wherein the author provides an interesting set of common edge cases. While they
use the same names or similar ones as those we’ve seen lately in class some
have new meaning here. In addition, they may have an actual value tied to the
name – such as zero, one, and two – but their values are defined otherwise.

 In order they
are as such:

  • Zero “represents
    any form of null input”, including the actual value zero but also null, an empty
    List or Array, et cetera. This of course is to test the capability of a program
    to deal with input that isn’t properly usable by it.
  • One represents what
    we have referred to as nominal up to this point, meaning a valid normal input
    which should test proper functionality of a program under ideal conditions.
  • Two does not
    correspond to the value two, but rather refers to testing the same code twice,
    usually in sequence, to see how repeated executions affect a system.
  • Two to Max-1 is
    most like one, in that it represents a nominal value as well, but in opposition
    this value should not be the absolute simplest needed to function but an average
    use case; meaning possibly complicated.
  • Max is
    fairly self-explanatory, used to test the upper limit accepted by a program, and
    can sometimes be an extreme value. As such, it can test the limits of the
    program under incredible load.
  • Max + 1 is
    used to ensure that limits placed on an application are working and that anything
    that does not correspond to a valid range is rejected in a reasonable manner.

These would most closely correspond to Normal Boundary Value Testing as we have covered in class but they are each less abstract than those counterparts. They provide an insight on what these values look like in actual QA testing, as well as expectations upon being used. Two for example does not have an actual value associated with it, but rather refers to a testing orthodoxy outlined above. One example of a test in this vein is found in another blog, in which the author proposes the edge case of the same user trying to log in from two different computers. I believe between the two of these blogs a clearer picture of the concepts we have covered can be found through these more concrete examples of the testing procedure.

Sources

A Beginner’s Guide to Testing: Error Handling Edge Cases
Build Strong Edge Test Cases

From the blog CS@Worcester – Press Here for Worms by wurmpress and used with permission of the author. All other rights reserved by the author.