One of the last types of testing techniques that we learned about this semester is known is Combinatorial Testing. Combinatorial testing is a testing technique that is used for software applications with a lot of different input possibilities and a high complexity. Even if you create a large number of different test cases, you will most likely still miss a test scenario. I’m not the best at explaining what something is, so I did some research in order to find an article that helps describe the aspects of combinatorial testing as well as how to use it and what its benefits are. This website is called testsigma.com

I think that this website does a great job at explaining what combinatorial testing is as well as all of its different benefits. In the article by Shanika Wickramasinghe, it states “Combinatorial testing is a testing method that uses multiple combinations of input parameters to perform testing for a software application. The main goal of combinatorial testing is to make sure that the software product can handle different combinations of test data as input parameters and configuration options.” This means that combinatorial testing takes a bunch of different input parameters, similar to pairwise testing, and uses it to test a bunch of different cases for the program. This can be extremely useful because some of the errors with a program can only be found with specific inputs. I’ve actually had this happen to me before in one of my classes. I wrote a program and testing a bunch of different inputs and they all worked, but when my teacher tried an input I never used, it failed. I think combinational testing is going to be extremely useful for me in the future. I know it seems like combinatorial testing can only be used in certain scenarios so it might be better to not learn it, but it actually has a lot of benefits according to Wickramasinghe:

• Covers a broad range of input combinations using a minimum number of test cases.
• Increases test coverage compared to normal component testing since it always considers multiple input combinations.
• Helps to detect bugs, defects, vulnerabilities, and unexpected outputs that might not be detected during the usual component and regression testing phases.
• Reduces testing effort, cost, and time. (Since combinatorial tests use fewer test cases to cover a wide scope of testing.)
• Identifies issues at the earliest while allowing the team to address and fix those earlier in the software development life cycle.
• Optimizes the testing process by removing unwanted test cases while ensuring that the cost and effort are not wasted on repeating the same test scenarios again and again.
• Helps to test complex software applications with a large number of parameters, settings, and options.
• Reduces the risk of critical defects going unnoticed, which can occur only when handling specific input combinations.

I recommend that everyone tries to use combinatorial testing at least once so they know how it works in case they ever need to use it again in the future to make sure all of their different input possibilities work.

Link: https://testsigma.com/blog/combinatorial-testing/

From the blog CS@Worcester – One pixel at a time by gizmo10203 and used with permission of the author. All other rights reserved by the author.

Throughout the cumulative experience I’ve had with testing, most of it focuses on the program’s logic and ensuring that it yields a correct result. One aspect of testing I have no experience with is security testing. Here one must find flaws within a system/program’s security and report them to developers so they cannot be exploited later in the product’s lifespan. Security testing has much higher stakes than that of unit testing as vital information such as consumer personal information and system source code may be leaked if there is a security breach. Therefore, testing security is of utmost importance when releasing a service to the public, as failing to do so will damage the service’s integrity.

Security testing hosts several different types of tests, each of which focuses on different aspects of a system. The article Security Testing posted by user pp_pankaj highlights the principles upheld by this testing and what each test achieves. Some of these tests, Posture Assessment, I found quite interesting. Posture Assessment combines the testing methods of ethical hacking, risk assessment, and security scanning into one report to provide an overall security posture of a system (pp_pankaj). Each of these subtests has a shared goal of having a hacker, hired by the development team, find security vulnerabilities within their system and report it to the team. Another form of testing I found interesting was social engineering testing. This deviates drastically from what we as programmers come to understand tests as. These are emulated attacks through communication such as email. The purpose of this test is to train developers to avoid suspicious engagement and to find new ways to breach a system without making direct contact. Whether a development team gets successfully breached through the socially engineered test is dependent on the team’s understanding of who they must respond to. A few weeks ago I was researching a data breach that happened earlier this year at Microsoft. Hackers were able to control a testing account and had direct access to employees on the project. From here they were able to obtain information they naturally should not have access to. All of this occurred due to developers not knowing that they must not communicate with a testing account.

A general metric as to whether security testing is vital to a project is to consider whether your product is liable for holding personal information that is not your own. If this is the case, then it’s in the development team’s best interest to uphold their principle of confidentiality and integrity by running security tests throughout the lifespan of your product.

-AG

Source: https://www.geeksforgeeks.org/security-testing/

From the blog CS@Worcester – Computer Science Progression by ageorge4756 and used with permission of the author. All other rights reserved by the author.

Beyond Unit Testing, there are several ways one may go about testing a developed system. One of these ways is through combinatorial testing. While researching this topic I’ve seen a couple of steps that I’ve taken in testing that match with this testing method. For example, earlier this semester I learned about how behavior tables can help guide Unit Testing by showing what aspects of a program will be covered by one test. Combinatorial testing achieves a similar effect by first taking all possible inputs (from a pool of predetermined inputs) and then creating a set of tests that will test each unique combination from the pool. A source that helped me grasp this topic is Combinatorial Testing by Shanika Wickramasinghe. In this article, Wickramasinghe provides an example of how they would develop tests. It’s important to note that in this example only combinations are created, NOT permutations, meaning that overall much fewer tests would be needed to fulfill a combinatorial test. This does raise a question for future reading of whether there is such thing as “permutative” testing and how that and combinatorial testing differ.

Using combinatorial testing does provide benefits despite the time it may take to achieve a successful test. Combinatorial tests are all designed to try multiple inputs simultaneously, meaning that both single-fault and multi-fault assumptions will be made in a full combinatorial test. Once these tests are complete, the developer can better understand which inputs cause a problem within their code. Additionally, once the pool of potential inputs is determined, the tester will have a set number of tests they must conduct. These tests may find faults in the program that require specific input that the development team may not have accounted for. Through feedback such as this, the development team can resolve the bug and create ways of handling errors caused by unexpected input. These benefits do come with equally heavy drawbacks. Manual combinatorial testing is possible, however the testers may struggle with creating combinations from a larger input pool. A way this can be solved is by using an automated combinatorial tester. It’s important to note that this can be limited by how intensive the tests are on the hardware of the automated tester. Lastly, the combinations that the test may provide could be so random that it’s nonsensical to test such a thing. This becomes an issue of resources which will vary from developer to developer. Ultimately whether one uses combinatorial testing or not is up to the developer. There are some instances where the cost of conducting one of these tests would be beneficial to the development process, but this is not a “one size fits all” type of test. By using some of the team’s resources, whether it be labor or hardware-bound, combinatorial tests will yield meaningful results as to which areas of the program need further testing.

-AG

Source: https://testsigma.com/blog/combinatorial-testing/

From the blog CS@Worcester – Computer Science Progression by ageorge4756 and used with permission of the author. All other rights reserved by the author.

The blog post “Mocking Made Easy: Understanding Mockito for Java Unit Testing” describes what mocking is and how to use mockito for java testing. I chose this blog post because this semester we have covered mocking and its uses during our in class activities. I feel like this post made effective and efficient descriptions for mocking and mockito allowing readers to grasp a greater understanding of the topic.

The blog starts with describing the importance of unit testing in software development and highlights the challenges developers face when testing components that have dependencies on other classes or external systems. The blog focuses on the tool Mockito. Which as a solution, facilitates the creation of mock objects to mimic the behavior of real objects, enabling isolated testing of individual components.The blog focuses on the fundamentals of Mockito, explaining concepts such as mocks, stubs, and spies. A mock object simulates the behavior of a real object, allowing developers to define its responses to method calls. Stubs are similar to mocks but focus solely on returning predefined values rather than executing real code. Spies, on the other hand, are used to monitor real objects while still allowing their original behavior. In addition, the blog highlights Mockito’s usage through code examples, demonstrating how to create mock objects, specify their behavior using method chaining, and verify interactions between the tested component and its dependencies.The blog post emphasizes the importance of clear and concise test code, advocating for readable and maintainable test suites. Additionally, the post explores advanced Mockito features such as argument matchers, which allow for flexible verification of method invocations with varying arguments, and annotations for simplifying mock creation and injection. The author also discusses best practices for using Mockito effectively, including avoiding excessive mocking, preferring real objects over mocks whenever feasible, and refraining from mocking third-party code unless necessary. In conclusion, the blog provides a comprehensive overview of Mockito, offering practical insights and examples to help developers harness the power of mocking for robust unit testing. 

After reading this blog post, I feel like I would be better prepared for software testing or quality assurance. The descriptions of mocking and mockito were very helpful in solidifying my knowledge on software testing as well as teaching me new ways to utilize mocking such as using the mockito tool. If I ever am in the situation where I need to create tests and to utilize mocking I feel more comfortable and prepared than before I read the blog.

https://blog.machinet.net/post/mocking-made-easy-understanding-mockito-for-java-unit-testing

From the blog CS@Worcester – Giovanni Casiano – Software Development by Giovanni Casiano and used with permission of the author. All other rights reserved by the author.

In a previous blog post, I had talked about Test Driven Development, or TDD. Today, I’m going to introduce a different approach that aims to almost rectify the potential shortcomings of TDD, that approach being Behavior Driven Development, or BDD for short.

BDD can be described  as “a collaborative approach to software development that aims to bridge the communication gap between business and technical teams” with the core idea of creating a “shared understanding of the software’s intended behavior using concrete examples” (Test Guild).

“The process revolves around writing scenarios using the Given-When-Then format, which describes the preconditions (Given), the action or event (When), and the expected outcome (Then).” This is a format that can be easily understood regardless of what people specialize in. TDD involved writing test cases and coding based on those test cases which mainly involved the developers, testers, and those that are closely linked to the programming and technical development. BDD, on the other hand, can involve the non-technical, such as stakeholders and those from other departments on top of the developers and testers. It can be simply put as, “compared to test-driven development (TDD) which is developer-centric, BDD is a team-wide practice” (Test Guild).

The Given-When-Then format allows for less misunderstanding when it comes to what is required of the software. Developers may use names that are short and to the point to describe something but it doesn’t match the behavior that is desired. The same developer or others that have just started working on the code may simply go along with it not realizing that what is desired of the code is something more or something else entirely. By using this format along with full sentences describing exactly what the code should do, there will be less room for error, misunderstanding, and time wasted fixing the code down the line.

One of the difficulties that seems to arise with the implementation of BDD is the inclusion of implementation details in scenarios. This is because scenarios are meant to focus solely on behavior. Including implementation details is basically attempting to set something in stone; scenarios describe what is desired of the code and how developers achieve that can change many times. It ends up adding more work every time that detail has to be met or changed.

BDD is an interesting topic, it seems to be a direct upgrade from TDD but that isn’t always the case. Take a classroom environment for example, it’s a bit odd as we (the students) could be considered developers but what about the other roles in the process? Would the professor act or technically be a stakeholder? It’s a process that can be learned at any point but it seems it can be only truly put into practice in a real world environment. We can certainly take aspects of BDD into mind, the Given-When-Then format and basing development around desired behaviors seems to have little to no downsides for any situation. 

Source: https://testguild.com/what-is-bdd/

From the blog CS@Worcester – Kyler's Blog by kylerlai and used with permission of the author. All other rights reserved by the author.

The blog post “Software Testing Basics: Positive vs. Negative Software Testing” explores two fundamental approaches in software testing: positive and negative testing. I chose this blog post because this semester we have been taught a variety of software testing techniques and strategies. From this blog post, it has categorized some of the techniques we have learned into one of two categories mentioned, positive or negative testing. I found this useful as it also allows us to know easily when to utilize certain techniques for certain scenarios.

The blog begins by describing the significance of software testing in ensuring the quality and reliability of software applications. Testing is important not only to detect bugs but also to enhance user experience and maintain credibility. Positive testing involves validating the software’s expected behavior under normal conditions. Test cases are designed to verify that the system functions as intended when provided with valid inputs. This method aims to affirm that the software performs its functions accurately and efficiently. By executing positive tests, developers can gain confidence in the system’s reliability and usability. On the other hand, negative testing focuses on the software’s ability to handle invalid or unexpected inputs and conditions. Test cases are designed to provoke errors, exceptions, or failures within the system. This approach aims to uncover vulnerabilities, defects, or unforeseen scenarios that may compromise the software’s performance or security. Negative testing is crucial for identifying weaknesses and enhancing the robustness of the software.The blog emphasizes the complementary nature of positive and negative testing. While positive testing validates the correctness of the software’s intended behavior, negative testing uncovers potential issues that might have been overlooked. Together, they provide comprehensive test coverage and contribute to the overall quality assurance process.Moreover, the blog discusses various strategies and techniques for conducting positive and negative testing. For example, positive testing involves scenarios such as input validation, boundary testing, and functional testing, where the focus is on confirming the expected outcomes. While, negative testing encompasses techniques like boundary value analysis, error guessing, and stress testing, aimed at challenging the error-handling capabilities of the code.

After reading this blog post, I feel like I would be better prepared for software testing or quality assurance. The descriptions of positive versus negative testing in my opinion were very helpful in solidifying my knowledge on software testing as well as teaching me new aspects of it. As previously mentioned, the blog post was beneficial for teaching me to know when to utilize certain techniques for various scenarios.

https://www.testmonitor.com/blog/software-testing-basics-positive-vs.-negative-software-testing

From the blog CS@Worcester – Giovanni Casiano – Software Development by Giovanni Casiano and used with permission of the author. All other rights reserved by the author.

What is JUnit?

JUnit is a Java testing framework that simplifies writing reliable and efficient tests. It’s especially suited for testing Java applications and offers features like multiple test cases, assertions, and reporting. JUnit is versatile and supports various test types, including unit, functional, and integration tests.

JUnit and Testing Types

JUnit primarily focuses on unit testing but can also handle functional and integration testing. Functional tests evaluate the functionality of a system as a whole, while integration tests assess how different components of a system work together.

How Does JUnit Work?

JUnit works by allowing developers to write tests in Java and run them on the Java platform. It provides features like assertions to verify expected behavior, test runners to execute tests, test suites to group related tests, and reporting tools to analyze test results.

Benefits of Using JUnit

• Organized and readable code.
• Early detection and fixing of errors.
• Improved software quality.
• Increased efficiency in the testing process.

Getting Started with JUnit

To get started with JUnit, developers can access tutorials, documentation, and forums for guidance. Setting up a JUnit project involves installing JUnit in an IDE like Eclipse or IntelliJ IDEA, creating a standard test file, and writing test methods.

Writing Test Methods

Writing a test method involves adding annotations, method signatures, method bodies, and assertions. Assertions like assertEquals, assertNotNull, assertTrue, and fail are essential for verifying expected results.

Creating and Running Tests

Creating and running tests in JUnit requires opening the project in a testing framework, selecting the desired test classes or methods, and executing them. Debugging modes like JDWP and Standard Streams help identify and fix issues during testing.

Troubleshooting Techniques

Troubleshooting techniques include using debuggers, checking documentation and forums, and running tests regularly. Well-written tests follow guidelines like keeping them small, relevant, and well-organized.

JUnit’s Assertions

JUnit’s assertions play a vital role in testing by checking conditions and verifying results. Common assertions include assertEquals, assertNotNull, assertTrue, and fail.

Conclusion

JUnit is a powerful Java testing framework that helps developers create reliable and testable code. By incorporating JUnit into their development process, developers can improve software quality, increase efficiency, and ultimately enhance their Java development skills.

Source – https://www.headspin.io/blog/junit-a-complete-guide

From the blog CS@Worcester – CS: Start to Finish by mrjfatal and used with permission of the author. All other rights reserved by the author.