In the realm of software development, ensuring the quality and reliability of a software solution is paramount. One crucial aspect of this process is system testing. In this blog post, we’ll delve into what system testing entails, its process, types, tools used, as well as its advantages and disadvantages.

What is System Testing?

System Testing is a vital phase in software development, where the complete and integrated software solution is evaluated to ensure it meets specified requirements and is suitable for end-users. It’s conducted after integration testing and before acceptance testing, focusing on both functional and non-functional aspects.

System Testing Process

System Testing involves several steps:

1. Test Environment Setup: Creating a testing environment for quality testing.
2. Creating Test Cases: Generating test cases for the testing process.
3. Creating Test Data: Generating data for testing.
4. Executing Test Cases: Running test cases using the generated data.
5. Defect Reporting: Detecting and reporting system defects.
6. Regression Testing: Testing for side effects of the testing process.
7. Log Defects: Logging and fixing detected defects.
8. Retesting: Repeating tests if unsuccessful.

Types of System Testing

1. Performance Testing: Evaluates speed, scalability, stability, and reliability.
2. Load Testing: Determines system behavior under extreme loads.
3. Stress Testing: Checks system robustness under varying loads.
4. Scalability Testing: Tests system performance in scaling up or down.

Tools used for System Testing

Several tools aid in system testing, including JMeter, Selenium, HP Quality Center/ALM, and more. The choice depends on factors like technology used, project size, and budget.

Advantages of System Testing

• Ensures comprehensive testing of the entire software.
• Validates technical and business requirements.
• Detects and resolves system-level problems early.
• Improves system reliability and quality.
• Enhances collaboration between teams.
• Increases user confidence and reduces risks.

Disadvantages of System Testing

• Time-consuming and expensive.
• Requires good debugging tools.
• Dependent on quality of requirements and design documents.
• Limited visibility into internal workings.
• Can be impacted by external factors like hardware configurations.

Personal Reflection

This resource has equipped me with valuable insights into system testing, which I believe will greatly enhance my job hunting process in software development. Understanding the various testing processes, types, and tools will make me a more competitive candidate, allowing me to target roles that specifically require expertise in system testing. Additionally, knowing the advantages and disadvantages of system testing will help me assess potential job opportunities more effectively, ensuring alignment with my skills and preferences. As I have seen many open roles looking for Software Q&A applicants.

Source: https://www.geeksforgeeks.org/system-testing/

From the blog CS@Worcester – CS: Start to Finish by mrjfatal and used with permission of the author. All other rights reserved by the author.

I recently read quite a few blogs regarding test-driven development (TDD), with many of them referencing behavior-driven development (BDD). This left me curious to learn about BDD and how it was different from TDD. Phillip Rogers does a great job breaking down what BDD is, the three principles of BDD, and some examples of BDD with Gherkin in his blog: “Behavior-driven development principles and practices.” (https://blog.logrocket.com/product-management/behavior-driven-development-principles-practices/#:~:text=Behavior%2Ddriven%20development%20(BDD),%2C%20domain%2Dspecific%20scripting%20language.) 

What is Behavior-Driven Development?

Behavior-driven development (BDD) is a product management approach focusing on defining system behavior from the user’s perspective. It emphasizes user interaction, collaboration among stakeholders, and aligning the product with user needs. BDD is a test-first development method.

3 Principles of Behavior-Driven Development:

1. What the software could do: Discovering and understanding customers needs to avoid building the wrong features. Techniques like impact mapping help prioritize features based on customer value.

2. What the software should do: Collaboratively writing structured documentation (executable specifications) articulating user needs. This involves using scenarios and examples in a given-when-then format to describe user behaviors.

3. What the software does: Automating desired behavior based on specifications, writing code, and iteratively improving both code and tests. This aligns with the test-driven development (TDD) process of writing failing tests, writing code to pass tests, and refactoring.

Behavior-Driven Development practices:

Impact mapping: A visual technique that reinforces what user outcomes are most important and are therefore more important to the project.

Story mapping: A visual technique that is used to maintain an understanding of what specifications are needed for a feature.

Three amigos: Forming sub-groups with different skill sets to work together. This brings different perspectives and thought processes into groups that may not have been there otherwise.

Benefits of Behavior-Driven Development

BDD being focused on the users’ perspective gives an improved understanding of the users’ goals. The tests created for BDD are typically higher level tests covering user scenarios. This ensures a high test coverage. Other benefits include enhanced collaboration, code reuse, and reduced rework as required changes are more likely to be seen early on.

Behavior-Driven Development vs Test-Driven Development

BDD and TDD are both test-first development methods that require planning and understanding of the project before beginning development. So, how are they different? TTD is mainly focused on the functionality of a feature. Whereas, BDD is focused on the users’ experience with that feature. BDD is more focused on testing specific scenarios a user may encounter. One notable difference is the fact that a single developer can do TDD where-as the amount of insight needed to do BDD requires everyone from developers to stakeholders,

From the blog CS@Worcester – CS Learning by kbourassa18 and used with permission of the author. All other rights reserved by the author.

For this week’s blog, I decided to research security testing because we didn’t have the chance to go over it in class. While researching, I found a blog called “Security Testing: Types, Tools, and Best Practices” by Oliver Moradov. The article is split up into a few main sections: “What is Security Testing?”, “Types Of Security Testing”, “Security Test Cases and Scenarios”, “Security Testing Approaches”, “What Is DevSecOps?”, “Data Security Testing”, “Security Testing Tools”, and  “Security Testing Best Practices”.

The first section “What is Security Testing?” explains the definition of security testing then spits off into two sections that explain the main goals and key principles of security testing using bullet points to organize the information. Security testing determines if the software is vulnerable to cyber assaults and evaluates the effect of malicious or unexpected inputs on its operations. Security testing demonstrates that systems and information are safe and dependable and that they do not accept unauthorized inputs. It’s a type of non-functional testing that focuses on testing if the software is configured and designed correctly. The main goals of this kind of testing are to identify assets, risks, threats, and vulnerabilities. It gives actionable instructions for resolving detected vulnerabilities and can verify that they have been effectively fixed. The key principles of security testing are confidentiality, integrity, authentication, authorization, availability, and non-repudiation.  

The next section provides multiple sections that delve deeper into the multiple types of security testing. The examples provided are penetration testing (ethical hacking), application security testing (AST), web application security testing, API security testing, vulnerability management, configuration scanning, security audits, risk assessment, and security posture assessment. I knew about a few of these types of security testing. However, it was interesting to learn about API security testing and security posture assessments. It provided information like how APIs allow attackers to gain access to sensitive data and utilize them as an entry point into internal systems and the basics of what a security posture entails. 

The blog then discusses some important test scenarios like authentication, input validation, application, and business logic then provides other tests in a bulleted list. It then discusses the types of approaches ( white box, black box, and grey box testing) and a few useful tools.

The next section that I found very important was the section about best practices. The best practices mentioned were: “Shift Security Testing Left”, “Test Internal Interfaces, not Just APIs and UIs”, “Automate and Test Often”, “Third-Party Components and Open Source Security” and “Using the OWASP Web Security Testing Guide”. I knew about some of the practices like automating and testing often and testing often but I did not know about the Web Security Testing Guide (WSTG). I like the fact that the author provided a link to that resource as well. I think this blog is a great resource for those who want to learn about security testing. It is well organized and made me feel like I’m a bit better prepared to enhance security for future projects. 

From the blog CS@Worcester – Live Laugh Code by Shamarah Ramirez and used with permission of the author. All other rights reserved by the author.

With artificial intelligence increasing computational power and variability in usage, I wondered what advances were being made with AI. The tedious and repetitive aspect of test-driven development can sometimes leave the development process stagnating, so I was interested in how AI is changing the software testing process. This blog post, AI for Unit Testing: Revolutionizing Developer Productivity, by Philip Riecks, expands on how AI is improving the quality of our code and the productivity of our developers. 

The article highlights AI’s revolutionary steps in software testing and development. It discusses tools like IDE plugins that act as digital coding assistants and surveys from GitLab that show a significant increase in AI usage and demand for AI testing solutions. Philip explains the benefits of AI, which include streamlining test creation, boosting developer productivity, reducing developer fatigue, and many more. The article addresses why developers hate unit testing, highlighting the importance of it despite its tedious nature. It then gives an assortment of those tools with a small explanation of their specialization.

I found this article very enlightening, especially regarding the impressive abilities of AI-driven tools. My first thought when thinking of AI is to fear for developers’ jobs or worry about copyright infringement. It is nice to see that the focus of AI tools based on this article is to help developers by removing tedious tasks and allowing them to focus on improving the code. One of the sections mention AI’s ability to use user stories to generate test cases automatically. This was particularly interesting to me because a big part of behavioral-driven development involves using user scenarios when developing tests. Having AI take the workload off those using the BDD method would significantly increase productivity.

While reading, I still worry about the experience of those who use AI. If AI predicts defect areas, creates tests, and assists you every step of the way, how will that affect your ability to do those tasks? I also wonder if it matters if our abilities are lowered if we always have the tool at our disposal anyway. I imagine it would end up the same way we use calculators. We learn and can do calculations, but use the tool for convenience. Overall, I’m cautiously excited about AI, the stress taken off developers’ shoulders, and the increased time they will receive to focus on enhancing their projects.  

In the future, I will endeavor to learn more about AI, focusing on current and upcoming tools. When I use these tools, I will use them as an assistant and not as a crutch. 

The Article:https://www.diffblue.com/resources/ai-for-unit-testing-revolutionizing-developer-productivity/

From the blog CS@Worcester – KindlCoding by jkindl and used with permission of the author. All other rights reserved by the author.

Unit Testing

Unit testing is a process in software testing where the smallest functional unit of code is tested. A common practice for unit tests is to first write the tests as code. Then the test code can be run automatically when changes are made. Doing this allows errors to be found, and isolated quickly if a test fails. Automating tests helps to ensure that efforts are focused on coding, rather than running tests.

As stated previously, a unit test is a block of code that verifies the accuracy of the smaller blocks of the code such as a function. The unit test checks that the block of code runs as expected, determined by the developer’s logic. A block of code may have more than one unit test assigned to it, in order to cover the full behavior of the block of tested code.

A block of code cannot be tested using a unit test because a unit test cannot use external data. Therefore the unit test needs to run in isolation. Because unit tests must run in isolation, this helps to improve the design of the code base to ensure that no function relies too heavily on other parts of the system. Doing this prevents code smells which can lead a rigid system due to insufficient modularization.

Unit testing strategies

There are strategies used when creating unit tests to ensure coverage of all test cases. Some of these strategies include error handling and numerous checks such as logic, boundary, and object oriented. Boundary checks and error handling are similar in the sense that they both check the behavior of the system based on inputs that are invalid/outside the expected input range. Object oriented checks ensure that the state of persisted objects are updated correctly. Lastly, logic checks ensures that the system performs as expected given a valid input.

Benefits of unit testing

A major benefit to unit testing is the increased efficiency when testing code and discovering bugs. Whenever the system code is changed, the same set of unit tests are run. If a test fails, it is easy to identify what caused the failure because tests are small and isolated. Therefore, unit tests help catch any bugs before the system reaches production. Another benefit to unit testing is unit tests act as another form of documentation. Unit tests act as documentation because developers can read the tests to see how to code should behave. Having accurate documentation is an important part of software, so that other developers know exactly what the expected behavior of the code is.

Conclusion

This article was chosen because I enjoyed that it explained when unit testing is less beneficial unlike other resources. This allowed me to understand when unit testing should, and should not be used. I enjoyed learning more about unit testing outside of class as it is such as integral part of software development. Therefore, I plan to implement unit testing in future projects to ensure system accuracy.

Resources:

https://aws.amazon.com/what-is/unit-testing/#:~:text=Unit%20testing%20is%20the%20process,test%20for%20each%20code%20unit.

From the blog CS@Worcester – Zack's CS Blog by ztram1 and used with permission of the author. All other rights reserved by the author.

Test-driven development is a method that talks about writing tests before writing code. It emphasizes how TDD has changed software development by making testing an iterative, ongoing process rather than an end phase near the project’s end. This blog talks about the TDD process, the roots of agile development, and its cycle of testing, code, and refactoring. As a computer science major understanding modern software development methods like TDD is important. I chose this source to gain more information about TDD’s principles, benefits, and best practices. Test-driven development’s approach to testing aligns with my idea of making sure that my software is high-quality. The source ideas of test-driven development integration with CI/CD pipelines also intrigued me, as it works with the best practices. It shows the idea of TDD, where each cycle starts with a clear testable goal, collaboration, and making sure it’s high quality throughout development. Learning about test-driven development agile principles highlighted the importance of flexibility, customer feedback, and adaptability in modern software development. One key takeaway was TDD’s role in enhancing collaboration between technical and non-technical stakeholders. By aligning development with user expectations and business objectives through clear, testable goals, test-driven development creates communication gaps and ensures that the software meets defined requirements. This topic means a lot to me as effective collaboration is needed for successful software projects.

The blog’s explanation of TDD’s benefits, such as improved design, and lower long-term costs changed the view of its value. Its emphasis on continuous improvement and code quality connects with my ideas of great software craftsmanship. The topic on best practices, including starting simple, writing expressive tests, and building an understandable test suite helps provide changes for applying test-driven development effectively. I plan on applying TDD’s principles and best practices in my future projects. Making sure I start simple, writing expressive tests, and building an understandable test suite are the steps I want to add to my skills. Additionally, integrating test-driven development into CI/CD pipelines for rapid and reliable deployments connects with software development best practices and emphasizes code reliability. This source not only has opened my understanding of TDD but also inspired me to embrace its principles in my software development skills. Reading and learning more about TDD’s principles, benefits, and integration with CI/CD pipelines is interesting. It has given me a deeper understanding of proactive testing, collaborative development, and efficient code delivery. Using a test-driven development cycle and best practices is not just a method. It’s an idea that fosters excellence and reliability in software development. This source has made me want to approach software development with a proactive testing mindset, ensuring quality and reliability in every line of code.

https://circleci.com/blog/test-driven-development-tdd/

From the blog CS@Worcester – Kaylene Noel's Blog by Kaylene Noel and used with permission of the author. All other rights reserved by the author.

Software testing is essential in software development, Static and Dynamic testing are two different approaches in software testing that are important in making sure the quality of the software is good. This source goes into detail about these two different testing approaches and their significance in ensuring top-notch software quality. It gives an understandable guide that breaks down the core concepts of both static and dynamic testing, highlighting their objectives, types, automation possibilities, and crucial decision-making factors. I chose this resource because it shows the idea of software testing in a way that’s easy to understand and the difference between static and dynamic testing. Static testing involves reviewing code and documentation before execution, aiming to catch any defects early in the development cycle. Dynamic testing focuses on validating software behavior during execution, ensuring it meets specified requirements and performs as expected in various scenarios. What interested me the most was when to use each approach and how it works in identifying errors. Static testing identifies coding errors early, contributing to improved code quality and easier maintenance. On the other hand, dynamic testing is important for assessing how software behaves in real-world scenarios, performance, and security.

I can see myself using what I’ve learned about static and dynamic testing, I could be using these testing techniques in my future projects and by having a choice on testing before or during the execution of the code. This source emphasizes the possibility of automation in dynamic testing, which aligns perfectly with efficient testing practices. Learning about automation opens up exciting areas for streamlining testing processes and enhancing overall software development efficiency. Static and dynamic testing are important in software development and testing. This source helps show the relationship between those two different approaches and how they contribute to building reliable, high-quality software. Both types of testing are good it just depends on whether you would want to test in the early stages like during code review or the execution of the code. Static and Dynamic testing techniques are vital in software development and they both complement each other by reviewing the softwares quality.

https://testsigma.com/blog/static-testing-and-dynamic-testing/#

From the blog CS@Worcester – Kaylene Noel's Blog by Kaylene Noel and used with permission of the author. All other rights reserved by the author.

JUnit is a popular Java testing framework that makes unit testing, test-driven development, and code writing more efficient for the development team. We discussed JUnit extensively in class. I wanted to discuss an article that provides a comprehensive guide to JUnit so those new to it can understand its fundamentals and importance to software development. This article, JUnit: A Complete Guide, by Shinji Kanaiis, is a valuable resource for beginners to JUnit and those who want a brief refresher. 

The article starts by highlighting the importance of reducing the number of defects early in the development process and how JUnit facilitates that through specific and efficient testing. It explains that JUnit can do this because its testing framework allows developers to write tests that target the functionalities of individual pieces of code. Without summarizing the article completely, I’ll briefly list its talking points. The rest of the article describes several aspects of JUnit, the most important being how it works, the benefits of using JUnit, how to install JUnit, JUnit assertions, and a step-by-step guide on making JUnit projects and tests. 

This article gave me an excellent refresher on what makes Unit Testing and, in turn, JUnit necessary. It is a modular type of testing that does not limit you to testing the entirety of your software but allows you to test individual classes and methods. It enables developers to fine-tune their testing process to find bugs efficiently. I also better understood its effectiveness when combined with test-driven development. Test-driven development focuses on testing first and developing code second. JUnit allows for better test-driven development because developers can efficiently test each case, method, and class until their code functions correctly before moving on to development. The article also talks about the vast resources and tutorials that JUnit has due to its popularity and ease of use. This aspect of JUnit reinforced my choice to write about it because it can be an excellent starting point for those new to Unit testing or software development. The explanation of assertions we discussed in class was enlightening, and there were some assertions that I was unaware of. This article was a great refresher on JUnit’s place in the development cycle and what makes it an excellent resource for software developers. 

I plan to use JUnit in future projects and make my code as modular, adaptable, and bug-free as possible. I will also point those new to JUnit towards this article or other resources so they can better understand how to get started.

The article: https://www.headspin.io/blog/junit-a-complete-guide

From the blog CS@Worcester – KindlCoding by jkindl and used with permission of the author. All other rights reserved by the author.

One of the last types of testing techniques that we learned about this semester is known is Combinatorial Testing. Combinatorial testing is a testing technique that is used for software applications with a lot of different input possibilities and a high complexity. Even if you create a large number of different test cases, you will most likely still miss a test scenario. I’m not the best at explaining what something is, so I did some research in order to find an article that helps describe the aspects of combinatorial testing as well as how to use it and what its benefits are. This website is called testsigma.com

I think that this website does a great job at explaining what combinatorial testing is as well as all of its different benefits. In the article by Shanika Wickramasinghe, it states “Combinatorial testing is a testing method that uses multiple combinations of input parameters to perform testing for a software application. The main goal of combinatorial testing is to make sure that the software product can handle different combinations of test data as input parameters and configuration options.” This means that combinatorial testing takes a bunch of different input parameters, similar to pairwise testing, and uses it to test a bunch of different cases for the program. This can be extremely useful because some of the errors with a program can only be found with specific inputs. I’ve actually had this happen to me before in one of my classes. I wrote a program and testing a bunch of different inputs and they all worked, but when my teacher tried an input I never used, it failed. I think combinational testing is going to be extremely useful for me in the future. I know it seems like combinatorial testing can only be used in certain scenarios so it might be better to not learn it, but it actually has a lot of benefits according to Wickramasinghe:

• Covers a broad range of input combinations using a minimum number of test cases.
• Increases test coverage compared to normal component testing since it always considers multiple input combinations.
• Helps to detect bugs, defects, vulnerabilities, and unexpected outputs that might not be detected during the usual component and regression testing phases.
• Reduces testing effort, cost, and time. (Since combinatorial tests use fewer test cases to cover a wide scope of testing.)
• Identifies issues at the earliest while allowing the team to address and fix those earlier in the software development life cycle.
• Optimizes the testing process by removing unwanted test cases while ensuring that the cost and effort are not wasted on repeating the same test scenarios again and again.
• Helps to test complex software applications with a large number of parameters, settings, and options.
• Reduces the risk of critical defects going unnoticed, which can occur only when handling specific input combinations.

I recommend that everyone tries to use combinatorial testing at least once so they know how it works in case they ever need to use it again in the future to make sure all of their different input possibilities work.

Link: https://testsigma.com/blog/combinatorial-testing/

From the blog CS@Worcester – One pixel at a time by gizmo10203 and used with permission of the author. All other rights reserved by the author.

Throughout the cumulative experience I’ve had with testing, most of it focuses on the program’s logic and ensuring that it yields a correct result. One aspect of testing I have no experience with is security testing. Here one must find flaws within a system/program’s security and report them to developers so they cannot be exploited later in the product’s lifespan. Security testing has much higher stakes than that of unit testing as vital information such as consumer personal information and system source code may be leaked if there is a security breach. Therefore, testing security is of utmost importance when releasing a service to the public, as failing to do so will damage the service’s integrity.

Security testing hosts several different types of tests, each of which focuses on different aspects of a system. The article Security Testing posted by user pp_pankaj highlights the principles upheld by this testing and what each test achieves. Some of these tests, Posture Assessment, I found quite interesting. Posture Assessment combines the testing methods of ethical hacking, risk assessment, and security scanning into one report to provide an overall security posture of a system (pp_pankaj). Each of these subtests has a shared goal of having a hacker, hired by the development team, find security vulnerabilities within their system and report it to the team. Another form of testing I found interesting was social engineering testing. This deviates drastically from what we as programmers come to understand tests as. These are emulated attacks through communication such as email. The purpose of this test is to train developers to avoid suspicious engagement and to find new ways to breach a system without making direct contact. Whether a development team gets successfully breached through the socially engineered test is dependent on the team’s understanding of who they must respond to. A few weeks ago I was researching a data breach that happened earlier this year at Microsoft. Hackers were able to control a testing account and had direct access to employees on the project. From here they were able to obtain information they naturally should not have access to. All of this occurred due to developers not knowing that they must not communicate with a testing account.

A general metric as to whether security testing is vital to a project is to consider whether your product is liable for holding personal information that is not your own. If this is the case, then it’s in the development team’s best interest to uphold their principle of confidentiality and integrity by running security tests throughout the lifespan of your product.

-AG

Source: https://www.geeksforgeeks.org/security-testing/

From the blog CS@Worcester – Computer Science Progression by ageorge4756 and used with permission of the author. All other rights reserved by the author.