Why DORA Metrics Are Just the Tip of the Iceberg for DevOps Team Productivity

Crowds of fun-seekers exploring a city on foot, "

Published on 4 November 2024 by Arjan Franzen

Imagine this: you’re a project leader, agile coach, or engineering manager looking for ways to make your software teams perform better. Not only do you want to make them faster and more efficient, but you also want to keep an eye on risks. How do you do that smartly? If you ask a techie on your team to measure productivity, they’ll likely come up with DORA metrics. These metrics provide a useful way to shed light on your software development process and measure productivity. But as with most metrics frameworks, is that enough, or should this manager measure more?

What Are Those DORA Metrics Anyway?

DORA Metrics are the secret sauce for measuring team performance. DORA metrics are the industry standard and were created by researching which metrics correlate strongly with high-productivity teams. DORA metrics are not just about speed but also about how stable and reliable your deployments are. Here are the four DORA metrics (or 4 Key Metrics, 4km):

Change Lead Time: This measures how long it takes for new code to go live.

Deployment Frequency: How often do you push new updates to production?

Change Failure Rate: How many changes introduce issues or bugs you must fix.

Recovery Time: How quickly do you fix things when something goes wrong?

These metrics will give you a good idea of how your teams are doing.

Are there any limitations or other restrictions to DORA?

While DORA is super useful, there is a catch. A bit like Murphy's or Goodhart's laws:

"When a metric becomes a goal, it's no longer a good metric."

If you only focus on these stats, teams can find tricks to improve scores without actually improving performance — that's not the point!

How to Get Started with DORA

Getting started with DORA doesn’t have to be complicated:

Collect Data: Start by collecting your current performance data. Tools like CI/CD logs are your best friend here.
Create Dashboards: Set up some flashy dashboards with tools like Grafana or Power BI.
Iterate But: Work in short, manageable sprints and keep checking your numbers continuously.

What Next? What to Do After DORA?

Once you’re up and running with DORA, it’s time to look further. Think about:

Talk to Your Teams: Interviews and surveys provide a deeper picture. Ask your people what they’re struggling with and what could be improved. Post-mortems and Retrospectives are great tools for this.

Gain Insight: Ask your developers for feedback on, for example, how smoothly the release process is going. Open-ended questions can tell you a lot. This often also results in new measurement points and insights that will be measured in addition to or sometimes instead of some DORA metrics. These insights are unique per organization and specific to these teams.

Combine Data: Link the figures, the DORA metrics, and the new insights to your BI platforms, such as Microsoft Power BI, Snowflake, or Data Studio, to see better context in the metrics and report to the rest of the organization.

Level Up! with Agile Analytics SaaS

The real magic can be found in platforms such as Agile Analytics SaaS. What can you do?

Set up Teams: Ensure that the information about teams and roles is linked to Agile Analytics.
Data Integration: Connect all relevant systems such as Jira, Gitlab, Azure DevOps, and CloudWatch to Agile Analytics and let the SaaS determine the service levels and make the dashboards available to the handy own UI or export to Grafana or Backstage.
Use Business Intelligence: Connect to platforms like Power BI, Data Studio or SnowFlake to gain deeper insights.

So here, in short, is a roadmap from the starting point via DORA to better teams and measuring what matters: happier and more productive Agile Teams.

Start small with DORA and Error Budgets, but with the assistance of our expert Agile Coaches, we’re confident we can supercharge your software operation together!