While good testing and version control habits are always helpful and save a lot of time, the process of committing changes, running tests, and deploying the changes can in itself be time consuming. This is especially true in an agile development process, which aims to make small, incremental changes in a series of sprints. Ideally, the process of committing, testing and delivering could be done in a single command.

This is where CI/CD comes in. An Oracle blog post on CI/CD describes the software development pipeline as being a four-step process: commit, build, automated tests, and deploy. CI/CD involves automating this process so that a single commit results in changes deployed to production, so long as the changes can be built and pass the tests. However, any of these stages can be skipped and not all of them have to be automated if it doesn’t make sense to do so. Additionally, other stages can be added if desired.

Oracle stresses the importance of testing in this process, and the requirement to have a suite of unit, integration, and systems tests. These are important to have for any project, but are vital if a commit is automatically deployed to production to make sure users are getting a reliable product.

Gitlab uses Runners to run the jobs defined in a file named “.gitlab-ci.yml”. Jobs can be defined in the stages mentioned above, or any arbitrary number of custom stages, to create a single pipeline. When a commit is made, the pipeline lists the stages and whether they pass. Of course, unit tests should be run before a commit to ensure that the code is behaving as expected, but once a change is committed the rest of pipeline can be automatic.

In large systems, this automation is especially useful. You may be not able to compile an entire code base on a local machine. You may not be working on the same schedule as the rest of your team. Automating other portions of testing and deployment allows a single developer to finish a feature or patch on their own and reliably get it into the hands of users.

In researching software engineering jobs, it seems that many companies use Docker as part of their CI/CD process. Imagine you want to make sure your software runs on Mac and Windows machines. Docker can use predefined images to build isolated containers in which the code can be run. A pipeline can contain separate build stages, for example, to make sure the code builds on both operating systems before running tests and deploying. Next week, I will look at Docker in more detail and describe how it can be applied to software testing.

From the blog CS@Worcester – Inquiries and Queries by ausausdauer and used with permission of the author. All other rights reserved by the author.

Writing software is fun. Mostly.

When I look back on the projects I’ve made, a couple big ones stick out which were mostly not fun. For example, I made the mistake of trying to build an Android app before I fully understood what a class is. Design patterns were not even something I had considered. The consequence was a fairly simple app with code that was supremely complicated and confusing.

For my next project, I tried to do better with the lessons I had learned. I continuously had errors that I didn’t understand; errors that were seemingly unrelated to the changes I had made. After a couple weeks of struggle I deleted all my files and uninstalled Android Studio. The frustration was so bad it made me want to quit programming. The frustration could have been prevented.

This is where tests, and specifically Test-Driven Development (TDD) could have saved me.

James Grenning has a blog post discussing how he feels about TDD, after years of writing software without it. TDD forces you to think through what you want your code to do. If you can’t test your code it’s likely a sign of bad design, so TDD indirectly leads to a better software architecture. Tests describe what the code is supposed to do, and the tests will always be there. If the code stops doing it in the future the tests will fail, acting as a permanent, enforced documentation. This leads Grenning to bring up the concept of regression tests, which confirm that your new changes don’t modify legacy behavior. The suite of regression tests that you slowly build over time will make sure nothing is inadvertently broken. But as you write new tests, you’re also getting instant feedback that your new code is performing to specification.

My life as a programmer is much better with TDD.

James Grenning

TDD brings you back to the programming exercises you did when you first started learning. It is satisfying as it is to struggle through a problem and make your project do what you want. But it is so much more satisfying to see a big green bar that tells you everything is working, including everything you’ve done since the beginning of the project. Then you can play with your new features manually if you so desire.

There are some downsides to TDD. More code means more time and more to maintain. Sometimes you’ll have to change tests because what your code is doing also changes. Furthermore, TDD is an art in itself and must be learned, meaning you may need to break old habits of bad design to write code that is testable. I’m inclined to agree with Grenning that debugging a complex system is much more time consuming that these downsides.

It only took a couple weeks of clear thinking to realize I had overreacted when I uninstalled Android Studio. I learned to have a lot more patience and take the time to do things the proper way. I still forgo testing and TDD for small, personal projects, but the benefits of TDD greatly outweigh the downsides for any important project.

From the blog CS@Worcester – Inquiries and Queries by ausausdauer and used with permission of the author. All other rights reserved by the author.

Programming was extremely hard for me at first. Once I discovered that programming isn’t just typing code, I spent so much time worrying about getting the perfect design, imagining what could go wrong, and researching alternatives that I rarely actually coded anything, at least for a brief period. While this concern may have taught me a lot in my research, I know better now. There’s a trade-off between time spent designing and time actually making software work.

Likewise, there is a trade-off between creating high-quality software and adding more valuable features, as Martin Fowler discusses in this blog post. Fowler is a software developer and author who has written extensively on the topic of software quality and design. In this blog post, he poses the question “is high quality software worth the cost?”.

Short answer: yes it is. To go further, Fowler believes that the question really doesn’t apply to software because high quality software is cheaper. He also makes the distinction between internal and external quality in software. For example, users notice a quality user interface, but have no idea which design patterns were used.

Internal quality is the goal of quality assurance and testing, and while it isn’t directly visible to the user, it makes it so much easier to provide a user with additional features by adding to an existing code base. In my last post, I wrote about SOLID and a podcast by Coding Blocks. They discussed how good patterns take a lot more code to write in the beginning, because things are broken up into smaller classes, with more, smaller methods. But as Fowler describes, the initial work more than pays off in the long run when code is easy to read and understand.

Anyone who has learned to program has spent hours looking for a mistake on a small project. These get harder to find as a project grows, especially when there are hidden dependencies among poorly-formatted code. Now imagine you wrote the code 2 years ago. And you can’t even fix it because you’re on vacation, but it’s preventing users from accessing their accounts so your coworker has to. Quality code and tests will drastically improve the chances that the problem is quickly patched and your users are happy. Take it one step further: this problem could have prevented entirely with sufficient testing and more focus on quality.

Most time is spent reading code rather than writing it. Fowler mentions this, but it is also an important theme in Clean Code by Robert C. Martin [1]. If you write quality code now, you can write even more quality code in the future.

[1] Martin, Robert C., et al. Clean Code: a Handbook of Agile Software Craftsmanship. Prentice Hall, 2016.

From the blog CS@Worcester – Inquiries and Queries by ausausdauer and used with permission of the author. All other rights reserved by the author.