The "Second Way of DevOps"

Author: Oliver Hankeln

Date:

Find out where you are missing feedback loops, while thinking outside your team or department. Implement these feedback loops, then amplify them, and use information radiators to create transparency.

Introduction

The last part of this series covered the First Way of DevOps as described by Gene Kim. This time we get to know the Second Way.

The Second Way

The Second Way is called “Amplify feedback loops”. This time it is not about the flow of work but about the flow of feedback. If you are already working in an agile environment you’ll have a lot of feedback loops to support you. In Scrum we have the events Sprint Planning, Daily Scrum, Sprint Review, and Sprint Retrospective; in pair programming we have constant flow of feedback between the driver and the navigator; other examples include, but are not limited to, code reviews, static code analysis results, or failing automated tests. Yet often we tend to only look at feedback loops within teams or departments. We want to go above and beyond that. How does feedback from operations get to the developers? How does feedback from clients get to operations? What are we getting feedback about? Where don’t we get feedback yet?

Why?

Continuous feedback helps us to keep an overview of what is happening. At the same time it helps to avoid that small problems grow into big catastrophes. A common understanding of the overall situation helps to instill a feeling of belonging within the organization. And we need good feedback as the basis for every empirical improvement process.

How do we do that?

Like the First Way, the Second Way does not offer a one-size-fits-all solution. We need to look for and find solutions specific to each organization. But there are some typical questions that can help to find your own way.

What do we get feedback about?

Start by collecting all the feedback that you have available right now. Is this information available to the rest of the organization or is it restricted to specific individuals, teams, departments, or branches? What data do other teams have available that could be of interest to you? The question is not whether you need that information, but whether you are interested in it and it might help you improve the status quo.

At a company I worked for we had live sales data on screens that were mounted at the walls. This was not strictly necessary for software developers, but it created an understanding of what it would mean for the company if we managed to reduce release downtimes or what an outage would do to the company. Once you know who would benefit from having access to what kind of information, make sure this information is available to them. Then collect ideas about feedback that you would like to have but that is currently unavailable. Think about how to get this data. As always, think about more than just your team or department.

How long does it take to get the feedback?

The rule here is: the faster, the better. The feedback is there to help you react to changing situations. Feedback you have to wait for is wasting valuable time.

If you only build your project once per night you will only get feedback about commits breaking the build once per day. If you manage to move towards Continuous Integration, you are improving the feedback loop and you will be able to use your time more effectively and efficiently. Short feedback loops are one of the reasons why you should try to avoid too long Sprints.

How do we get the feedback?

Feedback is only useful if it gets noticed and understood. There is no value in accumulating metrics that no one ever looks at. Instead, think about how you share the information with people inside (and maybe outside) your organization. Self-service solutions that can be used by anyone to search for information have proved quite valuable. But your key metrics deserve even more visibility and attention. If a build fails, send emails, activate red warning lights, etc. Use information radiators, e.g. put up displays at locations where they will be seen by many people. This could be the lobby or other common rooms. Show your most important dashboards there. They will be conversation-starters with people from different areas of work. These conversations will generate new ideas and help foster a better understanding between teams and departments.

Think about how you display a certain information. Displaying current hard disk usage, for example, is an interesting piece of data. But often the first derivative of this metric - the change in hard disk usage - is even more relevant and helpful.

What about all our precious secrets?!

Everyone within your company should have the ability to access all the information that is interesting to them. Doesn’t this pose the risk, that your competition could also get access to this data? There may be secrets that have to stay secret - and even secrets that not even the own employees may know about. Obviously, don’t show these on your screens in the cafetaria.

Yet, we should try to create and foster a culture of trust. In his seminal book “Team of Teams”, general Stanley McChrystal describes how he held daily briefings with hundreds of participants from many agencies for his anti-terror efforts. From his point of view this move from “need-to-know” to a common situational awareness was the precondition for his successful missions in the Middle East. So, be brave and trust your employees. Also, don’t overestimate the value of your data for your competitors. The gain in the ability to react and the common understanding usually far outweighs the risk that some information falls into the hands of someone they don’t belong to.

Conclusion

Like the First Way, the Second Way is a way without an end that you could reach. No matter where you are at the moment, you can always think about what the next small step to improving your feedback cycles might be. There will always be something worth improving. The only important thing is to start improving. And then continue to do it.