Is your team ready for DevOps?

July 31st 2015 Ian Buchanan in DevOps, Agile

Seven years ago at Agile 2008, Patrick Debois and Andrew Shafer were the only people interested in Agile Infrastructure. This year at Agile 2015, DevOps is a full-fledged track. Just like conference tracks, many teams concurrently pursue the perceived benefits of Agile and DevOps. Unfortunately, they often do so without building a shared understanding of the journey ahead. The Agile Fluency model has already been useful to help align team and business expectations. What can we apply those ideas to help the DevOps cultural movement avoid the same missteps?

Agile 2015 in Washington D.C.

By Barb Ballard on FreeImages.com licensed under FreeImages.com Content License.

Planning a DevOps journey

At OOP 2015 in Munich, I attended a talk by Steve Holyer about the Agile Fluency model. Agile luminaries Diana Larsen and James Shore proposed the model and summarize:

Agile methods are solidly in the mainstream, but that popularity hasn't been without its problems. Organizational leaders are complaining that they're not getting the benefits from Agile they expected. ... [This model proposes that] Fluency evolves through four distinct stages, each with its own benefits, costs of adoption, and key metrics.

Steve's talk helped me understand the gap between expectation and reality. Namely, people who acclaim Agile tend to lump all the benefits together. In reality, there are distinct stages, each with different kinds of investments and benefits. The journey toward DevOps has a similar gap. We hear the success stories from Flickr, Netflix, and Etsy. We relate to the starting point and aspire to where they are today; yet, we often miss the intermediate stages. I propose the Agile Fluency model can be mapped into DevOps concepts:

Change team culture leads to orient to value for your business. Change team skills leads to automate as a team. Change org structure leads to mesure business value. Change org culture leads to optimize the whole.

1. Orient to value for your organization

To the extent that DevOps is an extension of Agile, the 1st stage is the same for both. Your team needs to know what the organization values and plan in those terms. Working on the right thing is a benefit to everyone. What members of the team recognize as purpose, managers perceive as greater visibility. Orientation to business value means managers have enough information to redirect people and teams when business priorities change. Team goals require trust in management and other team members. That's why this stage requires a team be ready to change the culture of itself. Scrum and Kanban both provide good guidance and structure for making this kind of change. It should take 2-6 months for a team to become fluent in aligning to and tracking in terms of business value. If you go longer, maybe you need to try something else. Maybe Scrum isn't a fit for your team, try Kanban. Maybe going it alone isn't enough for your team, try an Agile coach. Or maybe the coach you have isn't focusing on the right things, try another.

There are many Agile practices that can help. I'm quite fond of Impact Mapping, User Stories, and Specification by Example. Each of those contribute turning big business goals into smaller pieces of work that a team can deliver in small chunks. In the Atlassian tool set, I reach for Confluence and JIRA to help. Confluence makes setting those business goals a collaborative process then keeps those goals ever-present as the team's target. Once oriented to business goals, JIRA Agile helps track progress.

Goal, Actor, Impact, Deliverable is the archetype for an impact map. 1M Players, Players, Invite Friends, Semi Automated Invites is a feature example. 1M Players, New Players, Drop off while waiting for provisioning, Pre-provision pool of unowned instances is a non-functional example.

One of the differences for DevOps in this stage is applying these practices to operations. Compared to development, operations often has more unplanned work such as responding to system outages. We have fewer industry examples about applying the same Agile planning practices to non-functional operating characteristics like reliability, performance, serviceability, and security. Just do a Google search on "Why DevOps" and you'll find all kinds of generalized justifications. However, DevOps is not a business goal in and of itself. It needs to fit the unique goals and direction of your business. Shipping your product faster is not the same as Etsy deploying 50 times a day. Shipping your service with better quality is not the same as Netflix maintaining sub-second mean-time-to-recovery.

2. Automate, as a team

In a previous role, I was on a brand new team. We had all written software, worked on Agile teams, and earned ScrumMaster Certifications. Despite all that experience, we still faced the frustration of relearning technical practices together as a team. Each one of us knew how to "do continuous integration" but we needed to learn what it meant to each other.

The 2nd stage of Agile Fluency addresses the mismatch between individual knowledge and learning-as-a-team. While the 1st stage requires a team culture change, the 2nd stage investment is a team learning technical practices together. You may need to attend workshops together. You may need a different Agile coach with technical chops. You may need to establish a "technical owner", as a parallel to product owner, to arbitrate decisions and prioritize automation options. With my team full of senior, agile developers we struggled with lowered productivity for a few months before reaching fluency as a team. I have known teams with no experience with agile practices to take as long as a couple years of practice before reaching fluency. Trying new things is risky. Avoid making it worse by starting too many new technical practices at once.

This stage also yields a different kind of benefit. Automation focuses on the delivery of value. Many Agile teams already focus capturing value frequently, by having 2 week sprints and "potentially shippable code" at the end. Frequency leads to revealing obstructions early. Yet, delivering value is more than just frequency. Delivery of value means putting control into the hands of the business so it can release software at the rate the market will bear. Management notices teams at this stage because they have low defects and high productivity.

Extreme Programming has long provided a bundle of Agile technical practices that apply to this stage: coding standards, unit tests built with test-driven development, automated acceptance tests, pair programming, serial and continuous integration, collective code ownership, and refactoring. I find it difficult to keep practices and tools separate in this stage. Except for pair programming and collective code ownership, automated tools closely match the Extreme Programming practices. I use IDEs and static analysis tools to enforce coding standards and perform refactoring. I use language-specific frameworks to make it easier to create unit tests and acceptance tests. Of course, both unit and acceptance tests are automation themselves. I use Bamboo for continuous integration to run all those automated tasks, as often as every commit. Good tools elevate the practice, making things more visible and aligned with business value. This is where software development turns in upon itself. Where economical, team practices become encoded as automation. This kind of automation is largely what developers bring to DevOps.

For some teams, those Extreme Programming practices might be enough to deliver value. For open-source and shrink-wrapped software products, done means packaged and tested. Other software becomes valuable once it's deployed to production. That's why Scott Ambler has advocated for thinking about operations in Agile. His Enterprise Unified Process advocated for Production and Retirement phases and for a discipline of Operations and Support. Yet it seems only recently that DevOps has drawn mainstream attention to the role operations plays in delivering value. Jez Humble and David Farley advocated for Continuous Delivery as a holistic view of delivering value. Their book extends automated testing and continuous integration into an automated deployment pipeline.

In the Atlassian tool set, Bamboo supports this idea by having specialized deployment projects. These model a deployment pipeline with environments, like Development, QA, Staging, and Production. Bamboo can track the deployment of an application into each of these environments. Although the building blocks of automation are often the same as continuous integration, tracking the pipeline requires a richer information model. Each environment serves an audience who needs to know how the automation works and when it is performed. In other words, Bamboo is approachable for not only developers, but also testers and operations.

Deployment projects expose which versions are running in which environments. Then explore test results for a particular version.

3. Measure business value

Before my experience on the "all star" Agile team, I had gone through many Agile transitions. At the outset, I always found it difficult to understand what the business wants, let alone measure it. As a backlog filled out, my team would turn toward leveraging automation to enable faster course corrections. Even at that point, measurement of business value always felt out of reach. This inward focus endured as long as the team was the constraint in overall delivery of value. When my team worked its way out of being the critical constraint, strange things would follow. We started waiting for the business to make decisions. Or for operations to provide us data about what we'd deployed. In 2011, Forrester coined this bookend experience, "water-scrum-fall". While software teams go Agile, business and operations often continue to operate with big-batch planning and execution.

Bringing business, development, and operations together goes beyond team culture and technical practices. This is a change in organizational structure. This stage has many of the benefits attributed to DevOps: better product decisions, higher value deliveries, and more reliability. Orienting to business value means you know what the business wants. It takes even tighter collaboration with the business to know how to measure the value of those things. Automating as a team means you know how to build "sensors" and aggregate data. It takes collaboration with operations to get to the right data. Fortunately, monitoring is a competency of most operations teams.

At Atlassian, our product teams measure Net Promoter Score and Monthly Active Users. It has been a long journey to survey and monitor not only our Cloud offerings but also on-premise products. Atlassian management uses these metrics to make business decisions. The objectivity and comparability of these metrics has replaced the politics with mutual trust. Product teams can negotiate rapidly to shift plans and align with what is best for the company as a whole. Data drives decisions at the speed product teams need them.

Of course, this kind of change typically takes a long time. The Agile Fluency model estimates 1-5 years. The cost is often measured in social capital expended on incorporating business and operations expertise into the team. The team will need to avoid the negative implications of turf wars. If you want the rewards of this level, start cross-team collaboration early, even as you are working on earlier stages.

With organizational change, there may still be some culture change and new practices. Lean Startup provides many good ideas for running experiments and measuring results. Lean Software Development also provides good guidance to help teams take a disciplined approach to optimizing value delivered through measurement. The tools for measurement are evolving. The scale of some organizations demands big data. However, many organizations don't need the most expensive and cutting-edge big data tools. Instead, they need to focus on different data gathering mechanisms. The key idea is logging. Everyone already has logs from running applications in production. So much log data accumulates that steps have to be taken to dump the overload of information. What makes logs so important is the time-centric structure. This time-centric nature is also reflected in business data warehouses, with time as a primary dimension. The dimension of time synchronizes business, development, and operations. As they say at Etsy, "If it moves, track it." The trick is finding the tools that can aggregate over the right time scale for your organization.

This report on build duration shows how long a build plan has taken over time. Is it getting slower or faster?

4. Optimize as a whole

If you are standing at or before the 1st stage, it can be hard to imagine "optimize as a whole". The Phoenix Project concludes with Parts Unlimited at the apex of this journey. Getting to this stage requires the biggest kind of change: culture change for the whole organization. There are few non-fictional guiding examples in the industry. The organizations that achieve this stage invest in inventing new practices, instead of following existing ones. The idea of optimizing as a whole means cross-team decisions don't require top-down authority. Teams that can measure business value well provide objective insight to the whole organization. That means teams can use data from other teams to optimize their role in the system. Agile Fluency recommends:

For most organizations, four-star fluency is probably best left as an aspiration for the future, at least until three-star fluency is within reach.

As individuals, I hope people set this as a target and reach for excellence. However, organizations need to balance risk and reward. The rewards of this stage are hard enough to describe and, with so few examples to draw from, the risks are even more difficult to enumerate. Although I count Atlassian as one of the companies with competency at this stage, I'll leave it to those who have mastered optimizing for the whole to offer advice.

Are you ready?

Is what you hope to gain recognized as valuable by the business? If not, seek first to understand. Then try again in the language of your organization.

Is your organization willing to make the investment? If not, set a more modest target of a lower stage. Then try again with a more focused option. Success will win the ability to try for the higher stage later.

Ready now? Try Bamboo. The deep integration with JIRA makes it ideal for orienting to business value. With out-of-the-box features for automated testing and deployment projects, you can start to involve QA and operations early. The rich information model and ease of use make it ideal for keeping the whole team involved.

Once you get started, remember the power of conviction. Forget that you can fail. Build the bridge as you walk on it.