For my last blog post for this class, I found an article online that
talks about the practice of shifting-left in software quality assurance.
This approach more or less emphasizes the importance of introducing quality
assurance to earlier phases in the development process. Testing from the
initial phase of development is supposed to prevent the amount of defects
and issues from piling up at the end of development. Having testing done
throughout the development phases can also lessen the workload for the
quality assurance team.

https://hackernoon.com/embracing-the-shift-left-approach-revolutionizing-quality-assurance-in-software-development

According to the article, the cost of testing and post-production vastly
outweighs the cost of development and planning. It posits that testing
earlier and more frequently catches bugs earlier on, and reduces the overall
cost of development. This goes very hand in hand with the agile software
development methodology we learned about last semester. The world of
software development has become much more fast paced, and the current
landscape pushes for finished products with minimal defects at
launch. 

I  have seen online the mentality that a product that ships with any
problems is often ostracized. Consumers want minimal issues and problems
when interacting with any kind of software, and that goes doubly for large
companies. Having software testers involved since the start of development
would allow teams a more seamless development experience.

One model for development that the article proposes has each stage of
development separated by a quality check gate, in which test cases are
implemented. When all defects are found and fixed, only then can the
development team move on to the next stage of the process. I think this a
very good system that could fit well within the agile sprint methodology.
Leave time at the end of the sprint, but before the sprint retrospective,
for the quality assurance team to check the code, then at the retrospective
they can sign off on the state of the project. If there are any bugs that
could not be fixed within this sprint, the testers can assign it as an issue
for the next one. 

During the Development Capstone project, this could be used to manage the
teams next semester. Have team members focus on quality assurance near the
end of the sprint, and then collect their feedback at the sprint
retrospective. Another way would be to have one or two teams act as quality
assurance throughout the whole semester. Either way it could save a bit of
headaches for everyone.

From the blog CS@Worcester Alejandro Professional Blog by amontesdeoca and used with permission of the author. All other rights reserved by the author.

For my final blog post for my Software Testing course, I wanted to go over something that also ties with other projects I’ve been working on during this semester in some way. In our Softawre Development Capstone team, we’ve had a couple of issues where we’ve had to touch upon the Thea’s Pantry continuous integration system through GitLab to have linter tests run automatically when pushing commits, so that commits that don’t match up to specification, the pipeline fails and the person who pushed those changes gets notified to fix any issues that arise. In this post, I want to go over a bit more detail about what continuous integration is and how it’s useful for developers.

According to Stephen Roddewig’s blog post on HubSpot, contiguous integration is an approach to development where code changes are regularly merged into a shared repository, built into a test application automatically, and the results of running this test application go back to the developer if any bugs or defects are found.

In practice, a continuous integration tool typically is an automated system where the source code is compiled with tests being automatically run on every individual push from a developer. This means that the developer’s contribution to the project is automatically tested on push, then either pushed forward to the maintainers or pushed back as a result of a pipeline failure.

The benefit of this is clear: bugs are caught quicker, the source code is updated on a regular basis with greater confidence because of the automated tests, and the pipeline provides an explicit and clear vision of what the specifications are for the project.

While working with the Thea’s Pantry system for my capstone, I could see these benefits in action. If a developer forgets to run tests locally (in the case of the Thea’s Pantry system, all tests are run in a script, and linters are run in their own script as well), the pipeline will catch any problems seamlessly, and the developer can easily see the pipeline failure, look at the output on GitLab, and determine what they need to fix in their branch.

In addition, it clarifies the specifications of what commits should look like, and what code should look like as well, on the basis that you can even add linters to the pipeline. It’s very useful in case someone forgets that we use conventional commits to have our changes be more clear with what they do, as the pipeline will detect that and function as a reminder of what things should look like.

All around, continuous integration is always a benefit for everyone involved in the software development process, creating a smooth system for testing that functions automatically rather than requiring developers to remember to run tests so they don’t mess up their branch.

From the blog CS@Worcester – V's CompSCi Blog by V and used with permission of the author. All other rights reserved by the author.

The blog post titled “Pairwise Testing: A Complete Guide” by Rajkumar (https://www.softwaretestingmaterial.com/pairwise-testing/) delves into the concept of pairwise testing, also known as all-pairs testing. I chose this blog because of its in-depth examples of  how to implement pairwise testing. These examples really made it easier to understand how pairwise testing can be used to reduce the amount of test cases while still maintaining high test coverage. The post also outlines the importance of pairwise testing and its advantages. Additionally, it compares pairwise testing with other testing techniques, highlights tools available for automating pairwise testing, and discusses its application in various scenarios.

What is Pairwise Testing?

Pairwise testing is a black-box testing technique used to reduce the number of test cases while still maintaining high test coverage. By focusing on combinations of input pairs, pairwise testing ensures that all possible pairs of input parameters are tested at least once. For example, if you had a program that took three inputs, X, Y, and Z, pairwise testing would create test cases for each combination of input pairs: (X, Y), (X,Z), and (Y,Z). At first glance this may seem like you have to create a ridiculous amount of test cases, but by following the steps for pairwise testing you can see how quickly the number of test cases is reduced.

Applying Pairwise Testing

The first step of pairwise testing is to identify all of your input variables. Next list all possible values for each variable. Variables that have numeric values may be reduced to valid and invalid to begin reducing the number of test cases needed. Then, by creating a table content columns of each variable and its possible outputs, you can get a list of each test case. 

This blog’s examples use tables to illustrate how each test case is created by filling in each column one by one.

The example from the blog uses a bookstore and creates a table of every variable. In this example you can see how each pair of variables covers all unique combinations except the book category and enquiry. These variables only test fiction with valid and non fiction with invalid. The other columns fixed this by simply reversing the order of one of the test pairs, but doing this would create the same issue for another combination of variables. In this instance two more test cases are added to the list to ensure the unique combination of book category and enquiry.

From the blog CS@Worcester – CS Learning by kbourassa18 and used with permission of the author. All other rights reserved by the author.

Like my previous blogs about performance and AI tools, I wanted to discuss another area of software testing that covers a specific development aspect. I chose security testing because while performance and functionality are essential, we must ensure our applications are safe for their users. This blog post, “Security Testing: Types, Tools, and Best Practices,” by Oliver Moradov, will outline various aspects of security testing, its importance, and its methodologies.

Oliver’s blog starts by defining security testing, listing its main goals and principles, and providing an index for many other aspects of security testing. It stresses that security testing is a non-functional testing process that determines the resiliency of software from cyber-attacks and data breaches. Non-functional testing focuses on how the system works rather than if it works. The primary goals are identifying assets, threats, vulnerabilities, and risks and performing remediation. 

The blog then discusses the types of security testing, including penetration testing, application security testing, API security testing, and vulnerability management. It also covers security test cases, and security testing approaches such as black box, white box, and grey box testing. The blog finishes by discussing best practices in security testing, such as shifting security testing left, testing internal interfaces, and automating tests.  

 I felt the distinction between functional and non-functional testing was important to discuss. Knowing how our data or the company’s data is handled is much more important for security than making sure the software functions correctly. It could function perfectly within the development team’s parameters but still fail to protect the company’s and users assets. 

I found it interesting that black box, white box, and grey box testing are applied differently than how they’re typically used in unit testing. The fundamental uses are the same, limiting and granting information, but there is much more to be gained when using this method in security testing. Testing what a malicious hacker could do with limited or abundant details on the software would be critical to protecting our assets. It also highlights the difference between non-functional and functional because developers use box testing to determine how to make errors occur within the system. In contrast, security tests would use the system functionality itself to create a breach in security. 

The sections detailing what security vulnerabilities they look for were enlightening. The authentication, input validation, and application logic sections highlighted what scenarios the security tests may encounter to validate system safety. I’m glad the blog discussed tools for developers to ensure their code can’t be exploited. Overall, this blog gave plenty of rudimentary information on security testing, and I urge those interested to check it out. In the future, I plan on using the tools and testing techniques to ensure that my code is safe for its users and owners. 

Blog: https://brightsec.com/blog/security-testing/

From the blog CS@Worcester – KindlCoding by jkindl and used with permission of the author. All other rights reserved by the author.

During the time I’ve spent in my software development concentration in my computer science studies (and even in general), we’ve mostly been concerned with the software development life cycle, where we focus mostly on getting a finished product as efficiently as possible that matches specifications, and build it up better and better over time. There is an interesting counterpart to this in the software testing life cycle, which is technically a part of the software development life cycle, but has it’s own specific steps.

In this post, I will be referencing this blog post from Testim on the STLC.

The point of STLC is similar to the point of SDLC at its core, getting a functional testing suitebased on specifications. The end goal has to do with finding problems and reporting them, however, rather than having a functional piece of software, which makes sense considering that testing is a step toward that piece of software.

The software testing life cycle is split up into 6 phases:

1. Requirement Analysis: Understand what the product should do, prioritize issues and brainstorm potential solutions (and whether they can be automated) with the team.
2. Test Planning: This is where the scope, tools and objectives are set for the following phases. It’s similar to a sprint planning meeting where tasks are assigned, time is estimated and issues are weighted.
3. Test Case Designing and Development: This is where the tests are, well, designed and created based on the specifications and priorities set up from previous phases.
4. Test Environment Setup: Software is ran on different configurations and setups to determine levels of performance and minimum requirements. We want to make sure our software works well on all possible configurations where it would be used, making a smoother experience for the end-user.
5. Test Execution: The tests are actually run all together, and the results are logged with details, and rerun with any changes to the main project as needed. Automated testing tools are preferred, as it makes this process significantly more refined.
6. Test Closure: Evaluate the testing result, taking into account things like test coverage, quality, and review the testing process. This is analogous to a sprint review, where the team comes together to review the results.

In an agile environment, these phases should all be covered in every sprint. All things considered, this is a necessary step in having working, quality software, as without a good testing environment your software could behave unexpectedly, and bugs will be more obfuscated.

From the blog CS@Worcester – V's CompSCi Blog by V and used with permission of the author. All other rights reserved by the author.

This post I will cover integration testing and why we use it today. Integration testing is a critical phase in the software development lifecycle, focusing on the integration of individual components into a cohesive system. It ensures that various modules or subsystems work together as intended. One of the primary challenges in integration testing is ensuring comprehensive coverage of interactions between different components. Identifying the right integration points and scenarios to test can be complex, especially in large-scale projects with numerous dependencies.

Selecting appropriate test cases to validate integration points is crucial. It requires understanding how components interact and designing tests to simulate these interactions effectively. Failure to cover all integration scenarios may lead to undetected defects, impacting the reliability and functionality of the software.

Moreover, integration testing often involves testing across different environments and platforms, adding to the complexity. Ensuring compatibility and consistency across various configurations is essential for delivering a robust product. One of the primary hurdles is achieving comprehensive test coverage across all integration points. Prioritizing critical integration points and designing effective test scenarios are essential to address this challenge.

Another challenge is managing the dependencies and external services during integration testing. Mocking or simulating external dependencies may be necessary to isolate various parts for testing, but it can also introduce its own set of challenges, such as maintaining realistic testing environments.

Furthermore, integration testing requires coordination among development teams working on different modules or services. Synchronizing changes and ensuring compatibility between components can be challenging, particularly in agile or distributed development environments.

Frameworks like Selenium are helpful for automating web browser interactions to test integrations between web components. For broader integration testing needs, companies might choose tools like Katalon Studio, which offers a comprehensive suite for web, mobile, desktop, and API testing. Additionally, some companies leverage enterprise-grade solutions like IBM Rational Integration Tester that provide robust features for complex integrations and compliance requirements. Ultimately, the choice of tool depends on the specific needs of the project and the company’s development environment.

Integration testing verifies the interactions between software modules, ensuring they function seamlessly as a unified system. Unlike unit testing, which examines individual components in isolation, integration testing focuses on how these components integrate and communicate with each other. It plays a crucial role in detecting issues arising from the integration of diverse elements, such as incompatible interfaces or conflicting behaviors. By identifying and addressing these issues early in the development process, integration testing helps prevent costly errors from surfacing in production. It’s step towards delivering reliable, high-quality software that meets user expectations and business requirements.

Blog Post: https://www.opkey.com/blog/integration-testing-a-comprehensive-guide-with-best-practices and https://www.testlearning.net/en/posts/integration-testing

From the blog CS@Worcester – Computer Science Through a Junior by Winston Luu and used with permission of the author. All other rights reserved by the author.

When I was in the process of learning how to build a PC as well as planning out what parts to choose, I had to weigh my options on the scales of price and performance. Some time after building it and using it almost daily, I was confident that my shiny new PC could handle nearly everything that I was using it for. I have yet to run into any major performance issues despite how my last sentence sounded but the fear of that still lingers in my mind when I try or want to try pushing my machine a bit. To prevent these worries, performance testing is done on software applications. 

Performance testing “is a type of software testing that ensures software applications perform properly under their expected workload.” It is meant to test and measure a system’s performance “in terms of sensitivity, reactivity, and stability,” using metrics such as response time, scalability, and resource usage (GeeksforGeeks). Doing so ensures that a system can handle the expected workload efficiently and effectively. 

Performance testing can be done on multiple levels such as on the application itself, the system running the application, and other levels that go a bit out of my knowledge.

There are many types of performance testing but the three that seemed most common and simple were load testing, stress testing, and endurance testing. Load testing involves testing the product’s performance under expected loads. This could include simulating user experiences through testing with different tiers of hardware, using the application for a certain amount of time, and doing things that a user may do. This ensures that the application is accessible to as many people as possible as hardware limits and mild activity will not do anything to the application nor the system. Stress testing involves putting the application under extreme loads, pushing to and even beyond its limits. This helps to identify the point in which the product breaks or affects the system, and gives an idea of what to optimize or fix in order to avoid breaking as much as possible. Endurance testing involves putting the application under stress for extended periods of time. Simply running the application over the course of a few hours, days, and possible weeks helps to identify memory leaks and how it handles a different type of load. 

Performance testing is really interesting to me as, on my PC building journey, I was very much interested in the performance of computers and how some applications require better and faster components to enjoy it at its true maximum. I, though not on the level of actual performance testing, have tested my own machine to see its limits, how it handles the maximum of certain applications, and how well it handles my use. Luckily, I’ve had very few crashing of applications, freezing, or other issues. 

https://www.geeksforgeeks.org/performance-testing-software-testing

From the blog CS@Worcester – Kyler's Blog by kylerlai and used with permission of the author. All other rights reserved by the author.

Security testing is a form of testing which is rapidly growing with consumers’ needs for security on the internet. As the article states, cybersecurity is becoming very important when you are online not only for personal reasons but also for business purposes. Many companies now use multiple online resources just to run their business day to day which may include payment systems, payroll, database management or any other range of services which are offered to companies through different platforms in order to help business owners run their business as efficiently as possible. These platforms have to be trusted so that consumers continue to use them and recommend them to friends, family, etc. Some examples provided of security issues within an  application are a student management system being insecure if the admission branch of the system can edit the exam branch, online shopping malls or any online storefront is not secure if the details from users credit cards or other payment methods is not encrypted as this opens up the users to credit card fraud, and even custom software can have security issues which is highlighted by the articles example of an SQL query retrieving the actual passwords associated with users accounts. Two tools recommended in this article for security testing are Invicti which is a web application that is used to scan both legacy and modern applications and Indusface which includes scanners for web, mobile and API applications. There are also different techniques involved with security testing some of these being access to application which involves going through the different roles in a system one by one ensuring they all only have access to what they should, data protection which similarly to access to application is meant to ensure that a specific user/role cannot see an aspect or menu of the application they are not supposed to and error handling which ensures detailed error messages cannot be used to aid in hacking.

Security testing while not being something we directly worked on during the semester it would have been interesting to work with as the many different types of security testing and the many different risks associated with application security. This type of testing seems to be much more manual in many aspects versus directly writing test cases as we did in most cases but being able to test for access or test the permissions of a role in a system would be very interesting to work with.

From the blog CS@Worcester – Dylan Brown Computer Science by dylanbrowncs and used with permission of the author. All other rights reserved by the author.