While writing about static testing, I spoke about my positive disposition toward dynamic testing due to its ease of access. Over the time spent writing the last blog In Defense of Static Testing, I grew to understand the benefits of testing with that approach. Now I seek to learn why dynamic testing may not be as “easy” as I once understood it to be.

One lesson I learned from the previous reading on static testing was the idea of ‘time and place’. Both testing methods are to be used under certain circumstances, rather than interchangeably. The blog post, Dynamic Testing, written by Swati Twade brings up this very idea. Twade states that due to dynamic testing’s cost, it ought to be used in effective ways. Furthermore, she warns that if dynamic testing is not used enough during the early stages of development, then there will be much more costly issues down the line. Here a certain balance must be found. The development team must use dynamic testing sparingly to reduce costs, yet must use it frequently enough to prevent larger issues from forming during the final stages of production. In these situations, static testing may be appropriate, as between dynamic testing intervals the development team can review the code and documentation to ensure that all aspects of the program are meeting their specifications. Naturally, if the team was only focused on these two aspects, then static testing could help in minimizing the use of dynamic testing however, dynamic testing captures much more than the code. According to Twade’s definition of dynamic testing, it seeks to capture both errors during the program’s runtime and how the execution of the software affects the hardware. These two focuses of testing cannot be captured by static testing, and require resources to use dynamic testing.

Through this reading, I’ve slowly begun to understand why both dynamic and static tests must be used under certain circumstances. Before researching this topic, I thought the two could be used interchangeably, without considering the resources it takes to conduct each test. Furthermore, each test focuses on reviewing specific content. Dynamic testing places high priority how on the program behaves once executed, meanwhile static testing places its priority on both the source code and documentation. Using these tests interchangeably without first understanding the conditions they work most effectively under will yield less than successful findings. Ultimately between these two blog posts, I have reinforced my knowledge of the testing processes, and have learned under what circumstances each thrives in.

-AG

Link to Original Post:

https://www.educba.com/dynamic-testing

From the blog CS@Worcester – Computer Science Progression by ageorge4756 and used with permission of the author. All other rights reserved by the author.

Recently in our course, we began an exercise in which all groups utilized static testing to find errors within a program we did not write. During this exercise, I could only think, “This would be so much easier if we could just run the code”. Rather than settling into this rut of compliance with instruction instead, I sought to find what makes static testing such a valuable approach.

Static testing is the approach to debugging where a developer (or group of developers), views the source code of a program and its related documentation. This is to find errors ranging from minor inconsistencies in naming conventions to bugs that will cause the program to break. Since static testing does not require the program to execute, developers can review the code and find errors while the compiler, among other tools, is being developed. Furthermore, all developers should be familiar with the language of the tested program, so the ‘barrier of entry’ is much lower than that of using new software to test/debug. Static testing when done in a group, can help question and apply design philosophies to the tested program. This not only helps reshape the program in the team’s ideal image but may also help developers within the group see how this design philosophy is applied.

The article What Is Static Testing, by Alexander S. Gillis, helps expand upon these aforementioned points in his work. What I found most interesting in this article was how he divides static testing into two approaches: review and analysis. Static review focuses mostly on documentation going over design, specifications, system requirements, etc. Static analysis focuses on the source code by finding errors within the structure and logic of the program. It’s important to note that both of these processes can be automated. Additionally, these types of testing, review and analysis, benefit most when used together and should not be a substitute for another. 

When done on a smaller scale, I can understand the importance of Static testing as it allows the developer to catch fundamental issues in logic or smaller bugs much earlier in the development process of the program. However, static testing may become much more convoluted as the program grows in scale. One issue that may stem from statically testing a larger program is the level of abstraction. If the code is poorly documented, or the intent of the program is unclear, then having a team review it may not be as effective. Ultimately static testing is most efficient when a team of developers understand most if not all aspects of the program, and are already familiar with both the documentation and code. If done manually, this requires much less resources than dynamic testing, allowing those resources to be used in other areas of production.

-AG

Link to Original Post:

https://www.techtarget.com/searchsoftwarequality/definition/static-testing

From the blog CS@Worcester – Computer Science Progression by ageorge4756 and used with permission of the author. All other rights reserved by the author.

For this week’s blog, I decided to do more research about pairwise testing. During my search for more information I found the blog “Pairwise Testing | What It Is, When & How to Perform?” by Kiruthika Devaraj. In this blog, Devaraj breaks down the process of pairwise testing in simple steps, along with the advantages and disadvantages. She first defines pairwise testing as a “permutations and combinations based software testing technique that tests each pair of input parameters to verify that the system functions correctly for all possible discrete combinations”. It uses a minimum set of test cases to test all possible discrete combinations. The purpose of this kind of testing is to test all possible discrete combinations of inputs without the need to test every single combination. This reduces the time, cost, effort, and number of cases that need to be written. It also allows the tester to identify errors that may occur when multiple different parameters are combined which in turn increases the overall quality of the software.

Devaraj then explains the disadvantages and challenges of this testing technique. One of the disadvantages that is explained is that it can be time-consuming especially if there are a large number of test cases. Another case is where assumptions can be false because pairwise testing relies on all pairs of parameters interacting to determine the results. She explains the challenges in bullet points to make it easier for the reader to comprehend. Some of the bullets include challenges in finding meaningful inputs and the correct set of inputs. If you don’t test carefully then the test coverage may not be enough. One also needs to determine how parameters interact with each other. 

The last few paragraphs of the blog describe using pairwise testing to speed up cross-browser testing. In this example, she explains the parameters that one would test (browser, operating system, device type, and screen resolution) and how to follow through with a real-world example. The author also includes a list of testing tools to help with automation for this type of testing.

The last section of this blog is the answers to frequently asked questions like “Is pairwise testing completely automatic?” and “How effective is pairwise testing at finding bugs?”. It helped clarify some aspects of the testing technique that weren’t explicitly answered earlier in the blog. 

Overall I think this blog was helpful in understanding pairwise testing especially because the author included a lot of examples. It gave me a better understanding of a testing technique that I may use in the future when I have to test a complex system that has multiple input parameters and multiple possible values for each parameter. 

From the blog CS@Worcester – Live Laugh Code by Shamarah Ramirez and used with permission of the author. All other rights reserved by the author.

TestFort published a blog (https://testfort.com/blog/difference-between-black-box-white-box-and-grey-box-testing#:~:text=While%20black%2Dbox%20testers%20make,in%20a%20non%2Dintrusive%20manner.) that does a great job breaking down each black-box, gray-box, and white/clear-box testing. By understanding each one individually, you can clearly see the differences between them. The blog also provides two other useful pieces of information for each of the software testing methods. Those being the most common design techniques and pros and cons.

Black Box Testing:

Testing is done by an independent team that has no knowledge of the code. The purpose of this testing method is to test the user interface’s functionality and usability. This testing method uses the users point of view to test the expected behaviors of the application.
The first example of black box testing that comes to mind is video-game testing. Testers are often given a demo of a game to play and give feedback on. These testers have no knowledge of the game’s code, however by playing the game they are testing for bugs and evaluating the user interface.

White/Clear Box Testing:

Testing is done by developers that have a great understanding of the application’s internal structure. This testing method’s purpose is to test and analyze that internal structure and its logic using the developers point of view. Developers commonly use tools to help analyze and debug code.
As I am only beginning my journey into developing tests I don’t have much experience with using debugging tools outside of ones provided inside some video games. However, when developing my own code I would commonly add print statements to verify variable values in the middle of the code. This would typically help me catch when values were being assigned incorrectly, scope issues, or simple off-by-one errors.

Gray Box Testing:

Testing can be done by developers, testers, and users with a similar testing approach as black box testing. Test cases are designed by an engineer that has decent knowledge of the internal structures. These test cases provide the testers with a glimpse into the software’s architecture without direct access to the source code. This form of testing aims to find defects and vulnerabilities in the structure of the code while combining the advantages of both black and white box testing.
This testing method is the one I have the least experience in, and is therefore the hardest for me to fully understand. My current understanding from this blog is relating it to when I would test software in the military. For obvious reasons, source code was hidden, however the functionality of each feature was documented so that testers could have a clear understanding of what the software was doing.

From the blog CS@Worcester – CS Learning by kbourassa18 and used with permission of the author. All other rights reserved by the author.

Are you a budding developer eager to dive into the world of software development methodologies? If so, you might have stumbled upon the term “Behavior-Driven Development” or BDD. Don’t fret if it sounds like a mouthful – we’re here to break it down for you in simple terms.

Behavior-Driven Development, or BDD for short, is a software development approach that focuses on defining the behavior of software through examples in plain English. It’s like telling a story about how your software should behave, but with a technical twist.

What’s the Buzz About BDD?

Imagine you’re building a house. Before you start hammering nails or painting walls, you’d want a clear blueprint, right? BDD acts as that blueprint for software development. Instead of diving headfirst into writing code, BDD encourages developers to first define the behavior of the software from the user’s perspective.

How Does BDD Work?

BDD revolves around three main players: stakeholders, developers, and automated tests.

1. Stakeholders: These are the people who have a vested interest in the software – clients, users, product managers, etc. In BDD, stakeholders collaborate with developers to define the expected behavior of the software through examples called “user stories.”
2. Developers: Armed with the user stories, developers translate them into executable code. But here’s the catch: the code is written in a way that aligns with the behavior described in the user stories. This ensures that the software behaves as expected.
3. Automated Tests: In BDD, automated tests are the guardians of software behavior. Developers write tests based on the user stories, and these tests are executed automatically to validate that the software behaves as intended. If a change in code breaks the expected behavior, the tests will catch it.

Why Bother with BDD?

BDD offers several benefits that make it an attractive approach for software development:

• Clarity: By describing software behavior in plain English, BDD makes it easier for stakeholders and developers to understand what the software should do.
• Collaboration: BDD encourages collaboration between stakeholders and developers, fostering a shared understanding of the software requirements.
• Quality Assurance: With automated tests validating the behavior of the software, BDD helps ensure that bugs are caught early in the development process, leading to higher-quality software.

Getting Started with BDD

Ready to dip your toes into the world of BDD? Here’s a simple roadmap to get you started:

1. Learn the Basics: Familiarize yourself with the core concepts of BDD and its key principles. There are plenty of beginner-friendly resources available online, such as tutorials and articles.
2. Choose a BDD Framework: BDD is supported by various frameworks and tools in different programming languages. Explore popular options like Cucumber (for Ruby and Java), Behave (for Python), and SpecFlow (for .NET).
3. Practice, Practice, Practice: The best way to learn BDD is by doing. Start by writing simple user stories and translating them into executable code using your chosen BDD framework. Don’t worry if it feels challenging at first – practice makes perfect!
4. Seek Feedback: Share your BDD efforts with peers and more experienced developers. Feedback is invaluable for improving your understanding of BDD and refining your skills.

Conclusion

Behavior-Driven Development may seem like a complex concept at first, but with a bit of practice and perseverance, you’ll soon find yourself harnessing its power to develop software with clarity, collaboration, and quality assurance in mind. So why wait? Start your BDD journey today and watch your software development skills soar!

References:

• Behave
• SpecFlow

From the blog Discoveries in CS world by mgl1990 and used with permission of the author. All other rights reserved by the author.

This week I had a bit of trouble thinking about what to write…until I lucky remembered what went on today. That being the Benchmark Trailer for Final Fantasy XIV: Dawntrail, a benchmarking tool that made sure your pc was up to specifications for the new expansion pack, but also tease some of the new gameplay, locations, animations and classes to come with it!

Which got me thinking…benchmarking is a great tool when it comes to testing not only hardware, but also software too! I think the two go hand in hand, “the two” being software and hardware, when it comes to testing. And then it hit me what I should write about: I should talk about what a benchmark is, and why it is important to testing!

For this, I consulted our favorite search engine, Google, and found this site!

https://testsigma.com/blog/benchmark-testing

So, what is a benchmark exactly? According to this site a benchmark “is a subset of performance testing that refers to a set of metrics or a reference point against which you can quality check your software or applications. The purpose of this testing is to compare the previous, present, and future updates of the application against a set reference.”

Now lets break that down a bit into digestible pieces…basically, every time you create a set of tests or even a set of functions for a program, you wanna make sure its always working, whether that be in general when its compiled, or on different device. Say you want to ship out a program to users on both Windows AND Mac. Youll want to create a benchmark environment where you can test and see how well the program functions on those.

There’s various ways you can do a benchmark, either by checking the programs functionality and tests on a Virtual Machine, testing it on hardware that differs from your own, or even running it on a frien’d computer as opposed to your own.

The reverse is true as well, as seen with the Final Fantasy Benchmark shown above. The test, while it sees if the functionality works on certain machines, it also lets you know if your machine/hardware is also unable to run the software. Sometimes your hardware might be so outdated that certain kinds of software are incompatible or unable to be run due to the older processing systems or runtime of the machine.

I am interested to see if we cover this at all in class, as it definitely seems like something that is a major part of testing when it comes to both hardware and software.

But thats all from me, until next week!

From the blog CS@Worcester – You're Telling Me A Shrimp Wrote This Code?! by tempurashrimple and used with permission of the author. All other rights reserved by the author.

Levels of Testing

Levels of testing refers to the different stages of testing software during the development cycle. Each level of testing aims to test specific aspects of the functionality of the software. The most common types of testing levels are unit, integration, system, and acceptance testing. Unit tests focuses on individual components, or units, of the system such as functions. Integration testing is used to ensure that these units work together as intended. System tests verify that the entire system meets the requirements specified by the customer. Acceptance testing is similar to system testing in that the whole system is verified. However while system testing tests the entire system, acceptance testing tests the final system.

Unit Testing

Unit testing is conducted at the code level, where each individual component is tested and the results are analyzed. Rather than manually testing each component, automating unit tests is highly recommended to reduce errors in analyzing and for increased efficiency. When making a unit test, an outline stating the expected results of the code should be made.

An example of an outline for a unit test for a calculator app that simply adds to numbers together would look like:

• Program should accept two numbers
• Program should return the sum of the two given numbers
• Program should handle negative values appropriately

Once the outline for the unit tests are made, then the actual code for the unit test should be written. The test will then check the code to verify it is doing what is expected once the test has run.

Integration Testing

Integration testing tests groups of individual components that are integrated into a system. This type of testing helps to identify any issue that may occur from coding errors or errors from the integration between units.

System Testing

System testing is performed on the application as a whole. All components are assessed against specific requirements made by the customer.

Acceptance Testing

Acceptance testing involves testing the final system. This includes the system’s functional and non-functional aspects such as performance, security, usability, and more. In this testing level, end users are given access to the software to ensure the end users’ needs are being met.

Conclusion

This resource was chosen because it clearly described the different testing levels, their purpose, and why each level is important. I enjoyed reading this article because it was written in a way that made understanding easy. In the future, when creating a complex system, I will be sure to follow the different stages of testing at the appropriate time in the development cycle to prevent issues later in development.

Resources:

https://testsigma.com/blog/levels-of-testing

From the blog CS@Worcester – Zack's CS Blog by ztram1 and used with permission of the author. All other rights reserved by the author.