07 - Platform Observability: DORA Metrics and SPACE Framework

What to measure about a platform, and what to leave alone

To know whether an internal platform works you need two families of measures. The DORA metrics tell you how well a service reaches production: speed and stability of releases. The SPACE framework reminds you that the productivity of those who develop cannot be reduced to a single number and proposes five dimensions, among them satisfaction. The former are computed from release logs, the latter also require questions put to people. This article shows how to compute the DORA metrics from a releases file (with an executed script), how to use SPACE to check that a dashboard does not measure a single thing, and which interpretation errors are documented.

A premise that comes from the DORA 2024 report, covered in article 1: whoever uses an internal platform turns out to be more productive but, in the survey data, has a throughput lower by 8% and a stability lower by 14%. That is why stability is measured from day one, and not halfway through the project. For service telemetry (metrics, logs, traces) I refer you to the Observability series, starting from the fundamentals and from Jaeger, Prometheus and Grafana: here I am talking about measuring the release process, not the application.

The five DORA metrics

The DORA metrics were born in 2014 as four variables, and since then they have changed name and number. DORA's history page tells of two changes: in 2023 the «mean time to recover» (MTTR) was redefined as the time to recover from a failed release, because the previous definition did not tell apart failures caused by a change from external ones, such as a data center outage; in 2024 a fifth metric was added, the rework rate. Today the five, in the official guide, are:

• Change lead time: the time from when a change is in version control to when it is in production.
• Deployment frequency: how many releases in a period, or the time between one release and the next.
• Failed deployment recovery time: the time to recover after a release that fails and requires immediate intervention.
• Change fail rate: the share of releases that require immediate intervention, generally a rollback or an urgent fix.
• Deployment rework rate: the share of unplanned releases, made because of an incident in production.

DORA divides them into two groups: the first three measure throughput (how many changes get through), the last two instability (how well releases go). The underlying message, repeated by the research, is that speed and stability are not in conflict: for most teams the metrics move together. The last two resemble each other and are read together: the fail rate counts the releases that broke something, the rework rate counts the releases that had to repair an incident. An urgent fix right after a release counts in both; an incident discovered after days counts only in rework, if it generates an unplanned release. This distinction is my own reading of the two definitions, not a text of the guide.

Computing them from your own data

To compute the metrics you need a release log with five pieces of information: when the change was made, when it went into production, whether the release failed, when the service came back to normal, and whether the release was an unplanned intervention. The following file lists fourteen of them, for two services, over two weeks. The data is invented and built with a contrast: one service that releases almost every day and one that releases twice in two weeks. The column names are Italian identifiers that the code reads as they are (servizio service, rilascio release, esito outcome, ripristino recovery, non_pianificato unplanned); the values failed and yes are the ones the code compares.

servizio,id,commit,rilascio,esito,ripristino,non_pianificato
checkout,c01,2026-10-01T09:00,2026-10-01T11:30,ok,,no
checkout,c02,2026-10-02T10:00,2026-10-02T14:00,ok,,no
checkout,c03,2026-10-03T09:30,2026-10-03T10:45,failed,2026-10-03T11:40,no
checkout,c04,2026-10-03T11:00,2026-10-03T11:40,ok,,yes
checkout,c05,2026-10-06T09:00,2026-10-06T16:00,ok,,no
checkout,c06,2026-10-07T10:00,2026-10-07T12:00,ok,,no
checkout,c07,2026-10-08T14:00,2026-10-08T15:00,ok,,no
checkout,c08,2026-10-09T09:00,2026-10-09T10:30,failed,2026-10-09T12:30,no
checkout,c09,2026-10-09T11:45,2026-10-09T12:30,ok,,yes
checkout,c10,2026-10-10T09:00,2026-10-10T11:00,ok,,no
checkout,c11,2026-10-13T10:00,2026-10-13T13:00,ok,,no
checkout,c12,2026-10-14T09:00,2026-10-14T10:00,ok,,no
accounting,k01,2026-09-23T09:00,2026-10-02T18:00,ok,,no
accounting,k02,2026-09-30T09:00,2026-10-13T18:00,ok,,no

The script reads the file and computes, for each service and for the whole, the five metrics. I wrote into the code the three definition choices that often remain implicit: the lead time starts from the commit and stops at the release in production; the recovery time is measured from the failed release to the recovery; the median replaces the mean, so that a few exceptional releases do not distort the number. The frequency is the number of releases divided by the fourteen days of the window.

import csv
import sys
from datetime import datetime
from statistics import median

GIORNI_FINESTRA = 14  # 2026-10-01 .. 2026-10-14


def leggi(percorso):
    with open(percorso, newline="", encoding="utf-8") as f:
        righe = list(csv.DictReader(f))
    for r in righe:
        r["commit"] = datetime.fromisoformat(r["commit"])
        r["rilascio"] = datetime.fromisoformat(r["rilascio"])
        r["ripristino"] = datetime.fromisoformat(r["ripristino"]) if r["ripristino"] else None
    return righe


def durata(minuti):
    if minuti is None:
        return "n/a"
    if minuti >= 24 * 60:
        return f"{minuti / (24 * 60):.1f} days"
    if minuti >= 60:
        return f"{minuti / 60:.1f} hours"
    return f"{minuti:.0f} minutes"


def minuti(a, b):
    return (b - a).total_seconds() / 60


def metriche(righe):
    n = len(righe)
    falliti = [r for r in righe if r["esito"] == "failed"]
    return {
        "releases": n,
        "frequency (per day)": f"{n / GIORNI_FINESTRA:.2f}",
        "change lead time (median)": durata(median(minuti(r["commit"], r["rilascio"]) for r in righe)),
        "change fail rate": f"{100 * len(falliti) / n:.1f}%",
        "failed deployment recovery time (median)": durata(
            median(minuti(r["rilascio"], r["ripristino"]) for r in falliti) if falliti else None),
        "deployment rework rate": f"{100 * sum(r['non_pianificato'] == 'yes' for r in righe) / n:.1f}%",
    }


righe = leggi(sys.argv[1])
gruppi = {"checkout": [], "accounting": []}
for r in righe:
    gruppi[r["servizio"]].append(r)
gruppi["ALL TOGETHER (do not use)"] = righe

for nome, rs in gruppi.items():
    print(f"[{nome}]")
    for chiave, valore in metriche(rs).items():
        print(f"  {chiave}: {valore}")

# Output (python3 dora.py rilasci.csv):
# [checkout]
#   releases: 12
#   frequency (per day): 0.86
#   change lead time (median): 1.8 hours
#   change fail rate: 16.7%
#   failed deployment recovery time (median): 1.5 hours
#   deployment rework rate: 16.7%
# [accounting]
#   releases: 2
#   frequency (per day): 0.14
#   change lead time (median): 11.4 days
#   change fail rate: 0.0%
#   failed deployment recovery time (median): n/a
#   deployment rework rate: 0.0%
# [ALL TOGETHER (do not use)]
#   releases: 14
#   frequency (per day): 1.00
#   change lead time (median): 2.0 hours
#   change fail rate: 14.3%
#   failed deployment recovery time (median): 1.5 hours
#   deployment rework rate: 14.3%

The third section of the output is the one not to use, and it is there on purpose. The checkout service has a lead time of 1.8 hours, a fail rate of 16.7% and a recovery time of 1.5 hours. The accounting service has a lead time of 11.4 days, no failures and a single release rhythm. The whole gives 2.0 hours of lead time, and says that «everything is fine»: it is the median of fourteen releases, twelve of which belong to checkout. DORA's guide puts it among the traps: the metrics apply to one application or service at a time, and mixing the contexts of different teams, or comparing a mobile application with a mainframe system, can be misleading.

The accounting numbers are not «bad» in themselves: with two releases in fourteen days any percentage is an illusion of precision (a single failure would bring the rate to 50%). That is why frequency and lead time must be read before the rates, and why a service that releases rarely is looked at over windows longer than two weeks.

The elite, high, medium and low levels: what they really are

The tables circulating with four performance levels are not fixed thresholds. The DORA 2024 report explains that the levels are not set in advance: they emerge every year from a cluster analysis of the survey responses. The 2024 comparison, on the four historical indicators:

• Elite (19% of respondents): lead time under one day, on-demand releases (several per day), fail rate 5%, recovery under one hour.
• High (22%): lead time from one day to one week, releases from one per day to one per week, failure 20%, recovery under one day.
• Medium (35%): lead time from one week to one month, releases from one per week to one per month, failure 10%, recovery under one day.
• Low (25%): lead time from one month to six months, releases from one per month to one every six months, failure 40%, recovery from one week to one month.

The table contains a datum that dismantles the idea of a single scale: the «medium» group has a failure rate (10%) lower than the «high» group (20%). The report observes that the medium group has a lower throughput and a higher stability than the high group, and infers that the level does not depend only on these two quantities. It also recalls that improving matters more than reaching a level: the best teams are those that improve the most, not necessarily those that stand highest. The checkout service in the example would stand, for lead time and frequency, close to the high groups, but with 12 releases a serious comparison cannot be made, and that is why I do not attempt it.

SPACE: why a single number is not enough

SPACE is the framework described by Nicole Forsgren, Margaret-Anne Storey and colleagues from Microsoft Research in a 2021 ACM Queue article. It has five dimensions: satisfaction and well-being, performance (assessed as outcome and not as output), activity (the count of actions and outputs of work), communication and collaboration and efficiency and flow. The article's thesis is that productivity cannot be reduced to one dimension or one metric, and the first myth it dismantles is that it depends only on activity: more activity can mean extra working hours to compensate for poor planning, or better tools that unblock the work.

The practical recommendations are two, and they can be checked with a program: capture metrics from at least three dimensions, and include at least one perceptual measure, that is a survey. The following script applies these two rules to two sets of metrics for the same team, adding a third rule that is mine (not all metrics should read the individual). The classification of the metrics into dimensions is also mine, as an example; in the article the «activity» category includes the count of commits, pull requests and releases, which is why releases appear as activity.

# Two sets of metrics for the same team. Dimension according to SPACE, type of measure,
# level at which it is read. The classification is mine, as an example.
activity_only:
  - {nome: commits per developer, dimensione: activity,    tipo: telemetry, livello: individual}
  - {nome: pull requests opened,  dimensione: activity,    tipo: telemetry, livello: individual}
  - {nome: coding hours,          dimensione: activity,    tipo: telemetry, livello: individual}

balanced:
  - {nome: production releases,                                    dimensione: activity,    tipo: telemetry, livello: system}
  - {nome: PR merge time,                                          dimensione: efficiency,  tipo: telemetry, livello: team}
  - {nome: perception of being able to work without interruptions, dimensione: efficiency, tipo: survey, livello: individual}
  - {nome: satisfaction with the tools,                            dimensione: satisfaction, tipo: survey, livello: individual}
  - {nome: change fail rate,                                       dimensione: performance, tipo: telemetry, livello: system}

In the file, each metric carries the SPACE dimension to which I assign it, the type of measure (telemetry or survey) and the level at which it is read (individual, team or system). The script that follows counts the distinct dimensions, looks for at least one survey and checks the levels, and prints a verdict for each set.

import sys

import yaml

DIMENSIONI = {"satisfaction", "performance", "activity", "communication", "efficiency"}


def controlla(metriche):
    dimensioni = {m["dimensione"] for m in metriche}
    errori = [f"unknown dimension: {d}" for d in sorted(dimensioni - DIMENSIONI)]
    if len(dimensioni & DIMENSIONI) < 3:
        errori.append(f"dimensions covered: {len(dimensioni & DIMENSIONI)} out of 5 (at least 3 needed)")
    if not any(m["tipo"] == "survey" for m in metriche):
        errori.append("no perceptual measure (survey)")
    if "individual" in {m["livello"] for m in metriche} and len({m["livello"] for m in metriche}) == 1:
        errori.append("my own rule: all the metrics read the individual, none the team or the system")
    return errori


with open(sys.argv[1], encoding="utf-8") as f:
    insiemi = yaml.safe_load(f)

for nome, metriche in insiemi.items():
    errori = controlla(metriche)
    print(f"[{nome}] {len(metriche)} metrics -> {'OK' if not errori else 'TO REVIEW'}")
    for e in errori:
        print("  -", e)

# Output (python3 space_check.py dashboard.yaml):
# [activity_only] 3 metrics -> TO REVIEW
#   - dimensions covered: 1 out of 5 (at least 3 needed)
#   - no perceptual measure (survey)
#   - my own rule: all the metrics read the individual, none the team or the system
# [balanced] 5 metrics -> OK

The first set is the dashboard you build by instinct: three activity measures, all individual. The script rejects it for three reasons, two of which come from the article (one dimension out of five, no perceptual measure). The second set touches four dimensions, includes two surveys and reads the three levels, and passes. The result says what a dashboard lacks, not whether the numbers are good, and it pays to keep few indicators: the article warns that too many metrics confuse and lower motivation. A note of caution: the article also proposes that metrics signal what the organization considers important, and that adding or removing them changes behavior.

Where to get the data, and when not to build the integrations

The data for the DORA metrics sits in three places: the version control system (when the change was made), the release pipeline (when it went into production and whether it failed) and the incident log (when the service came back to normal). The per-service perimeter, that is which services exist and whose they are, is given by the service catalog. Perceptual measures come from short surveys, repeated at a fixed cadence. A frequent temptation is to build right away the integrations with all the systems; DORA's guide warns that the initial investment may not be worth it and suggests starting from conversations, from DORA's «Quick Check» or from a product, open source or commercial, that offers ready-made integrations. The CSV file in this article is the minimal version of that collection, and for a small team it may be enough for the first months.

From the number to the improvement

The value of the metrics lies in the decisions they provoke. DORA's guide proposes a path in seven steps for a team that wants to improve:

1. Set a baseline with DORA's Quick Check.
2. Talk about the friction points in the delivery process.
3. Commit as a team to the most important constraint.
4. Turn the commitment into a plan, with more specific measures (for example how long code reviews last).
5. Do the work, changing the way of working if needed.
6. Check progress, with Quick Check, conversations and retrospectives.
7. Repeat.

For this path the order is part of the content: starting from the plan without a measured baseline and without having talked with the team produces improvements that nobody knows how to recognize. A recurring suggestion in the guide, for improving all five metrics, is to reduce the size of changes: small changes are easier to understand, to release and to undo.

The documented traps

DORA's guide lists the recurring errors. The most cited is turning the metrics into targets: a sentence like «every application must release several times a day by year end» ignores Goodhart's law and increases the probability that teams manipulate the numbers. Then: relying on a single metric, using the industry as an excuse (teams in regulated sectors saying that compliance prevents change), comparing different applications, keeping ownership of the metrics divided among development, operations and release (which produces friction and passing the buck), competing with other teams instead of improving, and favoring measurement over improvement. From the SPACE article comes one last warning: activity, by itself, must never be used to decide on the productivity of a person or a team.

What you can do now

You can list the five DORA metrics with their current definitions and you know why they are no longer four. You can compute them from a release log, per service, with the median and with the definitions written in the code, and you recognize that aggregated data from several services deceives. You know that the level tables are not thresholds, and that the 2024 «medium» group has fewer failures than the «high» group. You can check that a dashboard covers at least three SPACE dimensions and includes a survey. The qualitative listening to developers continues in article 9, while before that, in article 8, portability between clouds is tackled.