Performance Testing

Performance testing is the process of evaluating how a system performs under specific workloads. Responsiveness and stability of the system is monitored by examining the speed, reliability, and application size. Common indicators that are used for testing are network response times, number of users the system can handle at a time, processer memory consumption, and more.

Performance testing is an important part of the software development process. If an application is not tested before being released, users may have a negative experience because of errors that could have been found in the performance testing phase. Users that have a negative experience may be deterred from using the application in the future resulting in fewer overall users. Performance testing aims to find these errors before release, so users can have the best possible experience

When is performance testing done?

In software development, there are two main phases: development and deployment. Development testing focuses on individual components such as web services, APIs, and microservices. The earlier development testing begins, the earlier errors can be caught. Catching these errors early in the development phase is important because as the code base progresses, there is more and more code building on top of code that may have errors. This can make fixing the error more complicated. As the application becomes larger, comprehensiveness of the tests should also scale alongside. However, the application may get to a point where testing in the development phase becomes unreasonable and testing is done during the deployment phase. This happens when testing involves replicating a production environment, but recreating the environment is too difficult or expensive.

Types of Performance Testing

There are many types of performance testing that are done throughout the development process to verify the application performs as expected and can meet user requirements. Some of these performance testing types are:

• Load Tests – Tests the application under a realistic load by simulating virtual users. Response times are monitored, which can reveal potential bottlenecks to the system’s performance.
• Stress Tests – Similar to Load Tests, but the number of simulated virtual users is greatly increased. This is to see how the application runs when under peak activity.
• Soak Tests – Tests are conducted over an extended amount of time, as opposed to the tests mentioned above. This test is to evaluate how the system performs when under intense loads for a prolonged amount of time.

Conclusion

This article was chosen because it covered a variety of topics within performance testing, so I was able to gain a general understanding of what performance testing is. Performance testing is an important process in software development because it finds potential weak spots in a system. These weak spots can come from slow response times during peak activity, long load times, etc. Therefore, performance testing should be done on all systems to improve the user experience. In the future, I intend to look further into performance testing tools and frameworks to see how I can implement them in later projects.

Resources:

https://www.tricentis.com/learn/performance-testing

From the blog CS@Worcester – Zack's CS Blog by ztram1 and used with permission of the author. All other rights reserved by the author.

The three levels of the software testing process are unit testing, integration testing, and system testing.
Unit testing tests the isolated functionality of each individual unit. 
Integration testing tests the functionality of combining units to ensure they work together as intended. 
System testing tests the software as a whole. 

In my reading about integration testing, I noticed that at some point during this testing phase the system should be tested as a whole. Is system testing just a redundant and more in depth testing methodology focused on this? Sean Coughlin answers this question in his blog “Understanding System Testing in Software Engineering” (https://blog.seancoughlin.me/understanding-system-testing-in-software-engineering) by providing a clear understanding of what system testing is and what it tests for. 

What is System Testing?

System testing is a software testing phase that assesses a fully integrated software system’s compliance with requirements, covering both functional and non-functional aspects. These non-functional aspects are what makes system testing so comprehensive compared to integration testing when testing the software as a whole. Examples of non-functional testing include; performance testing, security testing, and usability testing.

Coughlin uses an e-commerce website as an example to illustrate system testing and each of its components. Functional testing ensures each feature of the website works as intended, Performance testing ensures the website still works even during peak user times. Security testing ensures the website is robust against cyber threats and that data is handled correctly. Usability testing ensures the website is user-friendly across multiple platforms.

Testing frameworks exist that allow for automated test such as simulating heavy loads for performance testing. User testing on the other hand is typically done manually as it focuses on a user’s experience. This means having a range of people, from developers to stakeholders, testing end-to-end use of the software. 

Overall system testing is testing an application for real world use. This goes much deeper than just functionality of the software. This is the last phase of testing before the software is released to the final product for acceptance testing. Meaning it is important that developers properly system test to identify and fix issues during this phase.

The scope for system testing is even larger than I could have imagined. Coughlin states, “I like to think of system testing as more of an umbrella term that captures lots of different testing forms.” which perfectly describes how system testing encapsulates testing all aspects of a software.

From the blog CS@Worcester – CS Learning by kbourassa18 and used with permission of the author. All other rights reserved by the author.

Another important topic that we have discussed in CS-443, or Software Quality Assurance and Testing, is known as Pairwise Testing. Pairwise Testing is yet another form of testing, but this type is a little bit different than the rest. Pairwise Testing, sometimes known as all-pairs testing, tests each pair of input parameters in order to make sure that the functions in the system run correctly no matter what the input is, guaranteeing that it will run for every combination. Pairwise Testing is known as a Permutations and Combinations (P&C) based software testing technique. A blog that I found to be really helpful in explaining Pairwise Testing is known as Pairwise Testing | What It Is, When & How to Perform by Kiruthika D. In the blog, she gives an example that helped me understand more. She states “Let’s say you have an application that allows users to enter two numbers, and the application will output the sum of the two numbers. You can use pairwise testing to test all possible combinations of two numbers, such as (1, 2), (2, 3), (3, 4), (4, 5), etc. By testing all the combinations of two numbers, you can be sure that the application is working correctly and will not fail when given different numbers.” This shows that you don’t actually test every single combination, but you test every single input with another input. This way, it makes sure that all of the inputs work instead of testing a potentially infinite amount of combinations.

The actual purpose/use of Pairwise Testing is exactly what I previously stated. It is used to make sure that all combinations of inputs are possible, but you don’t need to test every single combination. It can be extremely helpful as it reduces the amount of time it takes to test the program as well as the amount of effort. While Pairwise testing is a great testing technique, you obviously can’t use it all the time as it involves pairs. As for when to use it, Kiruthika states “Pairwise testing is helpful when testing complex systems that have multiple input parameters and multiple possible values for each parameter. It can significantly reduce the number of test cases that need to be created while ensuring that all possible discrete combinations of parameters are tested. This can help reduce test case creation time and cost and improve the software’s overall quality. Pairwise testing is not appropriate for all types of software testing. As we discussed, it is most effective for systems with multiple parameters and multiple possible values for each parameter. If a system has only a few parameters and a small number of possible values for each parameter, pairwise testing may be unnecessary. Pairwise testing, also, will not be useful if the values of inputs are inappropriate.” Essentially she is saying that Pairwise testing is used for functions that have multiple parameters with multiple values, and the order of parameters doesn’t matter. On top of that, depending on the type of parameter, the technique might not work either. While I personally don’t see myself using this technique in the future, I think that it has the opportunity to be very useful in certain situations, so I’m glad that I was able to understand it more in case I ever need to use it.

Link: https://testsigma.com/blog/pairwise-testing

From the blog CS@Worcester – One pixel at a time by gizmo10203 and used with permission of the author. All other rights reserved by the author.

For this week’s blog post, I decided to discuss the article “Unit Testing vs Test-Driven Development” by Albert Stec. I chose this article because it compliments the unit testing topic in the syllabus. This article discusses what unit testing is, why its important and how it fits into test driven development.

The first part of this article spends some time defining what a unit test is. “Unit test is usually a method that validates a small portion of the source code. So, the unit test is a programmatically written, automatic test. The unit test takes the initial data, passes it to the code under test, and asserts if the execution result is the same as the expected result.” The article then goes on to discuss what a well designed unit test should have.

The five most important parts of writing a good unit test according to the author are that it is fast, isolated, deterministic, readable and simple. “As we can see it looks simple. Although writing efficient unit tests could be complicated depending on the code we want to test. A well-written unit test should be: 1: Fast. A single project can contain a big number of unit tests, even hundreds, or thousands. Moreover, unit tests can be executed often, e. g., while developing a new feature to avoid regression or in CI/CD pipelines. Therefore, they must run as fast as possible. 2: Isolated. A unit test shouldn’t modify or depend on any external state. 3: Deterministic. A unit test should always return the same result no matter how many times we execute it. Of course, if nothing is changed between runs. 4: Readable. Unit tests are code that needs to be maintained. Therefore, it should be clear and easily understandable. 5: Simple. Often we can read that the unit test should contain a single assertion. Although it can be discussable, the fact is unit tests should validate small portions of the source code.” The next part of the article discusses the importance of unit testing in regard to software development.

This last part of the article discusses how unit testing is important to unit testing development. “To write appropriate unit tests before the logic developers spend more time on analyzing and understanding the problem and its domain. Therefore, the code is more likely to meet all requirements and clients’ needs. It’s one of the most important aims of TDD. The cycles play an important role, as the tests become more specific with time while the implementation becomes more generic.”

Article: https://www.baeldung.com/cs/unit-testing-vs-tdd

From the blog CS@Worcester – P. McManus Worcester State CS Blog by patrickmcmanus1 and used with permission of the author. All other rights reserved by the author.

The “TestGuild Automation Podcast” is a fantastic resource for computer science students preparing to enter the professional world. Hosted by Joe, the podcast dives deep into the realm of automation testing—a key aspect of software development where code is used to test other code. This method is vital as it helps detect and resolve issues quickly and effectively.

Joe has a unique ability to demystify complex testing concepts, making them accessible to listeners of all levels. His clear and straightforward speaking style ensures that even those new to the topic can grasp the essentials. The podcast is not just about learning; it’s an interactive platform where listeners can hear from seasoned professionals in the field. These guests share their journey, the challenges they’ve encountered, and the strategies they’ve employed to overcome them. This real-world insight is invaluable, as it shows the practical applications of theories and methods discussed.

The interviews with experienced testers and developers are particularly enlightening. They discuss not only technical challenges but also the dynamics of working in teams. This aspect is crucial in the tech industry, where collaboration and communication are as important as technical skills. The podcast sheds light on the collaborative nature of software development, emphasizing the importance of developers and testers working together to achieve the best results.

Here are some key takeaways from the “TestGuild Automation Podcast” that are especially beneficial for CS students:

• The inevitability of bugs: No matter how skilled a programmer you are, errors are inevitable. Testing is essential for finding and fixing these bugs before the software goes live.
• Creative problem-solving: Automation testing requires a creative approach to problem-solving, akin to the creativity needed in coding. Testers need to think outside the box to identify potential failures and vulnerabilities in software.
• The importance of teamwork: The podcast highlights how effective collaboration between developers and testers leads to the development of robust software. It’s a reminder that everyone’s role in a tech project is vital and interconnected.
• Career preparation: Understanding the basics of testing and quality assurance is a significant advantage in the job market. Employers value developers who are well-versed in these areas as it contributes to the overall quality and reliability of their products.
• Enhanced coding skills: By learning about testing, developers can anticipate potential issues and write cleaner, more efficient code from the outset.

For those who are not only interested in coding but also in the broader aspects of software development, the “TestGuild Automation Podcast” is an excellent resource. It provides a comprehensive overview of the testing landscape, offering insights that are crucial for any aspiring developer who wants to excel in the tech industry.

I highly recommend giving this podcast a listen. It’s not only educational but also engaging, making complex topics understandable and interesting. 

Link: https://testguild.com/podcast/

From the blog CS@Worcester – Site Title by Iman Kondakciu and used with permission of the author. All other rights reserved by the author.

Why I Like the “Code Review” Podcast and How It Helps Me Code Better

As a CS student, I’m passionate about finding ways to level up my skills. Lately, I was listening  the “Code Review” podcast, and it’s become an unexpectedly valuable resource. Honestly, I used to think code reviews were only for those super experienced developers, but this podcast has completely changed my perspective.

What the Podcast is About

Think of this podcast as your guide to making your code the best it can be. They discuss everything from finding those pesky hidden bugs to making your code easier for others (and future you!) to understand. They also dive into the specific rules and guidelines that professional coders follow. The best part is that they explain everything clearly and use real-world examples to make it relatable.

What I Learned and How It Changed My Coding

• Cleaner, More Reliable Code: I’ve picked up tons of tricks for writing code that’s well-organized and works the way it’s supposed to. This translates directly to better projects and way fewer frustrating “why-doesn’t-this-work?” moments.
• Becoming a Bug Detective: The podcast stresses the importance of having an eagle eye when reviewing code. I’m now way better at spotting potential issues early on, saving myself loads of time and headaches later.
• Understanding the Pro Mindset: Learning about the standards used in professional coding environments has been an eye-opener. It helps me understand what I’m doing well and where I can improve. This knowledge feels incredibly useful when working on school assignments and makes me feel more prepared for internships or jobs down the line.
• The Power of Collaboration: The podcast made me realize that code reviews aren’t about criticizing. Instead, they’re about teamwork and creating the strongest software possible. I’m excited about the idea of collaborating with other coders to make something great.
• Becoming a Helpful Code Reviewer: The hosts discuss helpful strategies for reviewing other people’s code, offering constructive feedback, and being a good coding teammate. I’ve become more confident in my ability to help others while also learning from their work.

Why I Recommend This Podcast

The “Code Review” podcast avoids overly complicated lingo, so it’s accessible even if you’re still learning the basics. I also love that I can listen while doing other things and still pick up incredibly useful knowledge.

You can find the “Code Review” podcast on most popular podcast platforms. If you’re eager to improve your coding skills, gain a deeper understanding of software development, and become a fantastic team player, I highly recommend checking it out.

Link: https://player.fm/podcasts/Code-Review

From the blog CS@Worcester – Site Title by Iman Kondakciu and used with permission of the author. All other rights reserved by the author.

As we transition into a time in which technology is an integral and nearly essential part of daily life, the necessity of security in the technology we use becomes ever so important. Apps, devices, websites, and everything in between may have access to your personal information. You may have entered passwords, names, bank information, card details, and much more when using these things and possibly even allowed them to save that information. You may have confidence that your information will not be leaked or stolen but even with the biggest, most trusted companies, it is not impossible. 

To combat the potentially massive amount of information from getting into the wrong hands, we perform what is known as security testing. Security testing is an “aspect of software testing focused on identifying and addressing security vulnerabilities in a software application… and… it aims to ensure that the software is secure from malicious attacks, unauthorized access, and data breaches” (hackerone). It ensures that users have their privacy, the system is safe and secure, and that the software meets regulations. 

Security testing involves identifying vulnerabilities in the software through means of scanning, hacking, and evaluating it. Hacking is a method that seems to stand out compared to the other likely due to how it’s unique to security. Hacking falls under what is known as Penetration Testing which involves simulating real-world attacks on the software in order to expose vulnerabilities. Hacking, by definition, is the gaining of unauthorized access to data in a system or computer, but in this case this form of testing falls under ethical hacking. Ethical hacking is essentially hacking but with legal authorisation and is intended solely for security reasons. Overall, security testing is the perfecting of a software’s security through means of identifying and exposing weaknesses and remediating them.

I’ve actually had my debit card info taken in the past and my money used to purchase things without my knowledge. Thankfully, I got all my money back as well as a new card but, at the time, I was extremely anxious and worried. I was desperately searching for what could have taken my information and to this day, I still have no clue. It was likely my fault for putting my information in some shady place but there’s a chance that a site or app I had my info in and trusted was breached. I believed that I was being very careful with what information I shared and where I shared it but that situation really shook me. Security testing is not a topic we went over in class but it is a good topic to learn about nonetheless. Although learning about the testing of security is not something that everyone is interested in, I believe that teaching people how to protect their information is becoming more and more important.

https://www.hackerone.com/knowledge-center/what-security-testing#:~:text=Security%20testing%20is%20an%20important,unauthorized%20access%2C%20and%20data%20breaches.

From the blog CS@Worcester – Kyler's Blog by kylerlai and used with permission of the author. All other rights reserved by the author.

The article “Property-Based Testing: A Test Strategy for Modern Software” on Codemotion’s website explores into the concept and benefits of property-based testing (PBT) in the context of modern software development. PBT is a systematic approach to testing software by defining general properties that the program should satisfy, rather than focusing solely on specific input-output pairs.

The piece begins by highlighting the limitations of traditional example-based testing, which often fails to cover all possible edge cases and scenarios. This leads to the introduction of PBT as a more comprehensive testing methodology. Unlike example-based testing, where tests are explicitly defined, PBT generates test cases automatically based on the properties defined by the tester.

The article talks about the key components of PBT, including generators, properties, and test runners. Generators are used to create random inputs for testing, ensuring a wide range of scenarios are covered. Properties, on the other hand, represent the general rules or invariants that the software should adhere to. These properties are then used to validate the behavior of the software under test.

One of the main advantages of PBT highlighted in the article is its ability to uncover edge cases and bugs that may not be caught by traditional testing methods. By generating random inputs based on defined properties, PBT can uncover unexpected behaviors and corner cases, leading to more robust and reliable software.

The article also discusses the challenges associated with PBT, such as the need for careful property definition and the overhead of generating and executing a large number of tests. Yet, it argues that the benefits of PBT outweigh these challenges, particularly in complex systems where traditional testing methods may fall short.

I really like this blog post because not only did it explain the theoretical aspects of PBT, the article provides practical examples and case studies to illustrate its effectiveness in real-world scenarios. It also showed like coding examples which really helped understand how it work instead of just trying to imagine how it could look like.

From the blog CS@Worcester – CS- Raquel Penha by raqpenha and used with permission of the author. All other rights reserved by the author.

The article “Pairwise Testing” by Ryan Craven explores into the concept and application of pairwise testing, a powerful technique used in software testing to enhance efficiency and effectiveness. Pairwise testing, also known as all-pairs testing, aims to analytically reduce the number of test cases required to test a system while still maintaining thorough coverage.

This blog begins by introducing the problem of combinatorial explosion in testing. Traditional thorough testing, where every possible combination of inputs is tested, quickly becomes impractical as the number of variables or parameters increases. Pairwise testing offers a solution to this problem by focusing on testing pairs of input values. The key insight is that many faults in software are caused by interactions between pairs of inputs rather than individual inputs themselves.

The author explains the core principles behind pairwise testing, emphasizing its ability to efficiently cover many combinations by selecting a representative subset. This approach significantly reduces the number of test cases needed compared to exhaustive testing while still providing effective coverage.

This blog also shares the concept of pairwise testing with a simple example involving a fictional coffee shop application. By identifying the parameters and their respective values (e.g., coffee size, type, extras), he demonstrates how pairwise testing can be applied to generate a minimal set of test cases that cover all possible pairs of values.

Also, the article discusses the benefits and limitations of pairwise testing. While it offers substantial reductions in test case count and provides good coverage, it may not detect faults involving interactions between more than two inputs. Craven advises on when pairwise testing is most suitable, such as in situations with limited time and resources or when dealing with complex systems with numerous input parameters.

This was a good blog post to read because it covered somethings we talked about in class during the group work. Through clear explanations and examples, the article serves as a helpful guide for understanding and applying pairwise testing in real-world scenarios. This is definitely something that will be very useful in the feature as my professional career expanded

From the blog CS@Worcester – CS- Raquel Penha by raqpenha and used with permission of the author. All other rights reserved by the author.