What engineering intelligence actually tells you — and what it can't

If you have been asked to "get some data on delivery", you will quickly find a category of tools that promises exactly that. You will also find it has no settled name. Engineering intelligence, engineering management platform, software engineering intelligence, SEI platform — these are largely the same product category, named differently by whichever vendor is doing the naming.

This is not a review of them. It is an attempt to answer the question underneath the shopping: what does this kind of data actually tell you, and what is it never going to tell you? Both halves matter, and the second half is the one that determines whether the tool ends up being used or quietly abandoned.

Strip away the positioning and these platforms all do the same three things. They connect to the systems where work already leaves a trace — Git, your ticket tracker, CI, incidents. They reconstruct the path a piece of work took through those systems. And they turn that path into measures: how long things take, where they wait, how often they come back.

That is genuinely useful, because the trace already exists and nobody has to fill in a timesheet to produce it. It is also the source of every limitation that follows. These tools measure the parts of the work that happen to pass through a system. A design decision argued out in a meeting, a dependency nobody owns, a team that has quietly stopped disagreeing with its lead — none of that appears, and no amount of dashboard will surface it.

Almost every purchase in this category is driven by one of three needs. They look similar in a demo and are quite different in practice, and knowing which one is yours saves you from comparing feature grids.

If your honest answer is the first one, be sceptical about whether you need a platform. A recurring query against the data you already have will produce one chart at a fraction of the cost, and the whole category is priced for the second and third jobs.

Feature lists in this category are nearly identical, so they are close to useless for choosing. Three things genuinely differentiate, and none of them are usually on the comparison page.

This is the section we would want to read before buying, so it is the longest one here. None of these are hypothetical; they are the common ways this goes wrong.

By far the most common outcome. The platform is bought, connected, configured, demoed once — and then nothing changes, because no recurring meeting has a slot where the data is looked at and acted on. The tool is not the hard part. The operating rhythm around it is, and it is the part that no vendor can install for you.

The diagnostic is simple: if delivery data disappeared tomorrow, would any regular decision be made differently? If not, you do not have a measurement problem yet.

Several tools in this category can rank developers by commits, lines, review volume or a composite score. Some market it. Our position is that this is a mistake, and it is worth being explicit about why rather than just disapproving.

Software delivery is a queueing system, and most delay in a queueing system comes from the structure of the queue rather than the speed of the workers. When you rank individuals you are attributing a system property to the people inside it. What follows is predictable: people optimise the measured thing, pull requests get smaller and more numerous, review gets rubber-stamped, and the number improves while delivery does not. You also lose the ability to hear about problems, because reporting one now costs you personally.

If you take one thing from this article: whatever you buy, keep the unit of measurement at team level or above.

Goodhart's law is not a clever aside here, it is the main operational risk. Put deploy frequency on a leadership slide and deploy frequency will rise — through smaller, emptier deploys if that is the cheapest way to move it. The measure stops describing reality at the moment it starts being used to judge people.

The practical defence is to pair every throughput measure with a quality measure and always read them together. Deploy frequency alongside change failure rate. Cycle time alongside rework. A team can game either one alone; gaming both at once is just doing the work properly.

The category makes it easy to collect two hundred metrics and use none. Decide in advance which two or three actually inform a decision, and treat the rest as diagnostic — available when you are investigating something, not on a dashboard, and never in a report to anyone outside engineering.

That last one matters more than it used to. Tools in this category do get retired — Pluralsight Flow is being wound down, and its customers are discovering that proprietary metrics do not survive a migration.

We measure delivery — DORA metrics, cycle time, error budgets — at team level, and we work alongside the teams using them. That combination is the reason to talk to us and also the reason not to.

If you want a self-serve dashboard you configure yourself and never discuss with anyone, several tools do that well and you should buy one of them. If the failure mode you recognise is the first one on this page — data collected, dashboards built, nothing changed — that is the problem we exist to solve, and the platform is the smaller half of it.

Worth reading next: how to measure software development productivity without the measures turning into targets, and whether developer experience needs metrics at all.