Importance of testing in DevOps in the age of continuous delivery

Fundamentally, DevOps helps shorten the time between an idea being coded and it being production-ready, and to do this consistently and quickly you need to carefully define and automate your testing. You must put the right processes and tools in place to make testing an integral part of the cycle.

As continuous delivery becomes the norm, expectations have increased amongst those not doing it. The ability to release good quality code quickly and support continuous feedback during delivery is absolutely critical. The role of testing is often overlooked in the DevOps process, and many underestimate the benefits automation and putting the right test tooling in place can provide. When thinking about all of these aspects, companies can more effectively release quality software in today’s age of continuous delivery. Continuous delivery is about being able to efficiently release code quickly and with a high degree of certainty that things are going to work. And work securely.

How can companies get DevOps working well?

One of the ways to get DevOps working well is to focus on automating testing. The path of testing units of code is already well-defined, though also looking at end-to-end user testing gets you further down the path. This takes away repetitive error-prone manual browser-based testing steps where people are examining a web site or application and trying to decide “does that look right?” or “has that regressed since last time?” or “has the associated change had the correct effect?” Instead of making a subjective judgement, you can use tools that look to automate those tests to a high degree of certainty.

According to Alex Zavorski, VP of Atlassian Products at SmartBear: “Another key component of modern software delivery workflows is continuous testing. The antiquated approach of leaving testing in a time-boxed, late-stage of your delivery lifecycle is impossible in true agile and DevOps at scale. Testing must be performed not just earlier, but continuously throughout the entire software delivery lifecycle.”

Below are three guiding principles for successful DevOps which are worth bearing in mind when thinking about the role and importance of testing:

Flow Feedback Learning

Enable rapid flow of ideas, code

and design through the system Find problems quickly, validate ideas rapidly, let the system take the work Provide a culture where we can learn and experiment, and fail fast

The role of testing

A key achievement in getting teams to work in an agile way is having your developers able to work in a culture of innovation. So, in development and in testing they might be saying “let's try this, let's test it and

let’s get some rapid feedback so that we can decide whether this was the right thing to do or not”. The DevOps angle on this is in enabling the team to get that work deployed and tested effectively. It's the same agile mindset, but at an operational level.

Ultimately, the cycle should move fast, so that code can be written and quickly put in an environment where the presumptions on which they were first written are rapidly validated or invalidated. Perhaps this involves testing out different hypotheses on different groups of users, or perhaps it’s just delivering a new feature or fix quickly. The key point is getting frequent flow into the system so you can learn as much as you can and then move to the next thing.

Benefits of automation

Companies often come from a scenario where a release manager is responsible for releases and they do their work manually. Every time they do a release, they follow a document which describes how to deploy the code in a sequence of commands and manual steps. Pre-release environments may also look different from production, thus leading to complex human decisions being made during releases. This is error-prone and time-consuming and doesn’t scale well when a company tries to increase the frequency of software releases.

Automation becomes essential when this time comes, and this, in turn, frees people up to do more creative and more knowledge-intensive work. It also lets you scale whatever you're trying to automate much faster than you could without it, without just hiring additional engineers to run more tests.

Putting the right test tooling in place

Once you’ve adopted a mentality of automating testing, you can begin unlocking all the automation potential in your organisation by getting people to use the right tools. And using the technology for the purpose it’s supposed to be used. The above diagram shows examples of tools for the various parts of the DevOps process. Examples of testing tools include Kitchen CI to test infrastructure - making sure that files and packages are in the right place and have the correct permissions, and Cucumber for acceptance testing to ensure that the code is working before it’s sent out.

We also saw that the coronavirus pandemic accelerated the growth of tools such as Slack acting as an information hub, making it easier for people to find the data that they need. According to Stewart Butterfield, the CEO of Slack, this is “a shift that’s inevitable over the next decade. And I think it just accelerated by a couple of years because there were also people who thought Slack was great and really enjoyed it, but essentially just used it in the way that they might have used AIM or Yahoo messenger or something like that 20 years ago. It was essentially for direct messaging. Who suddenly are beginning, depending on what they do, bringing in integrations with Salesforce, or marketing automation tools, or their HR system.”1

Finally, the COVID-19 pandemic highlighted the importance of tools that work remotely, especially when enabling more nuanced collaboration. DevOps teams are often distributed across multiple time zones - there is no 9 to 5 anymore and people are having to work remotely. This means that more than ever, having centralised, consistent and automated processes are vital for visibility; with testing the area where there is the most to lose by not making test data consistent and visible.

In conclusion

Increasingly in today’s remote working environment, the ability to integrate testing and ensure quality is critical to any business’ DevOps strategy and combining the right tools and automation to decrease time to market is increasingly important. Implementing effective testing into your DevOps strategy gives you the confidence that you can release quickly, leaving you confident that your customers are getting the value they need from you.