How do other companies measure their AI Software Factory?

Key Takeaways

• The companies measuring AI engineering well track what their factory lines are producing and whether that output holds up, not just how many engineers have access.
• Most companies start with velocity: the share of tasks, PRs, or reviews an agent handles. It’s easy to count and easy to explain to leadership.
• Velocity alone misleads. Acceptance rate can’t show a bug an agent missed. A PR count can’t show whether the code held up in production.
• The three measures that give a complete picture are velocity, quality, and cost, read together, not in isolation.
• Define what you’re counting before you report it. “Agent-authored PR” can mean very different things depending on whether a human edited the result, and the gap between definitions can be large.

Why Measuring AI Impact Is Hard

Ask ten engineering leaders whether their organization has adopted AI, and most will say yes, 70% of teams have adopted AI developer tools widely or completely. Ask whether that adoption has moved the needle, and the answers get less certain. In LeadDev’s AI Impact Report 2026, 68% of respondents cited a lack of clear metrics as their top AI challenge and only 31% effectively measure the impact of AI.

That’s surprising for an industry built on agile. Software teams adapt to new tools and methods all the time. So what makes measuring AI impact so different?

• It changes the tools and the method at once. Teams are changing what they use and how they work simultaneously. Existing engineering metrics can still show that delivery got faster or slower, but they can’t tell you whether AI was the reason.
• It’s a category, not a tool. Measuring AI is less like evaluating one project management vendor and more like evaluating all of SaaS. AI shows up in nearly every tool, service, and workflow in your stack, so there’s no single place to measure it.
• It moves work around. AI can speed up one part of the process while slowing down another. Junior engineers may open more pull requests while senior engineers slow down their review, so a gain at one end can be offset by a loss at the other.

With so many tools, models, and workflows in play, no single number can capture AI’s impact. Adding AI adoption metrics for every tool you buy only multiplies the problem. A better approach is to treat your SDLC as a single AI software factory and measure its output as a whole: how quickly work ships, how often it breaks, and what it costs. That’s how the companies below approach it.

What Top Companies Are Measuring their Software Factories

There are as many ways to measure AI impact in your software development process as there are ways to implement it. The companies below have each published explanations of how their software factories work, and we compiled them in Inside the Software Factory. Together, they show engineering intelligence practices that have adapted to an agentic SDLC and moved beyond AI adoption and usage metrics.

One interesting insight Uber and DoorDash have published isn’t a number; it’s which numbers they stopped relying on. Uber retired “developer years saved” because it made engineers anxious about being replaced, its baseline kept shifting, and business leaders prioritized revenue and feature impact instead. DoorDash stopped trusting its own acceptance rate once it realized it could never show a missed bug, so it built a benchmark to check it. Both kept the numbers that answered a real question: whether the agent’s work held up. Velocity, quality, and cost, read together, get you closer to that answer than any single headline figure.

The Metrics Teams Start With

Uber and DoorDash started from opposite ends: Uber measured usage across the entire system, while DoorDash measured the quality of a single job.

Uber started with adoption and spend. Its cost formula begins with users and sessions per user, the two metrics it wants to keep growing, then multiplies them by the work agents do and the price of tokens. These are AI adoption metrics with a price attached.

DoorDash started with one job and a baseline. It picked code review as the first use case for its in-house cloud-agent platform because it was frequent, measurable, and easy for engineers to assess. Its earlier third-party reviewer converted 46% of settled high- and critical-severity findings into code changes, setting a benchmark for DoorDash to beat.

Others started with share of merged PRs. Ramp and Shopify report about three in four and one in eight, respectively, as the number of merged PRs their agents write or coauthor. 

It’s important to remember that AI has changed engineering intelligence and delivery metrics, not replaced them. A deployment is a deployment, regardless of who wrote the code. As Port’s guide notes, agents can inflate throughput, so pair DORA metrics like change failure rate with rework rate. For more, see Measuring the real impact of AI on your SDLC.

Four Ways to Measure an AI Software Factory

Each metric answers a different question: how fast agents move work, whether that work held up, and what it cost.

1. Cost per session breaks total AI spend into its parts: users, sessions per user, turns per session, requests per turn, tokens per request, and price per token. Tracking cost per session separately from cost per request shows where money is going and which part of the chain to fix. Uber cut cost per session by 52% from June to August 2026 by reducing turns and requests per session while allowing user growth to continue. (Inside Uber’s software factory, Measuring agentic impact and ROI)
2. Spend per engineer makes the cost visible at the person level rather than only in aggregate. Uber shows engineers a live cost counter and sends Slack alerts at 50%, 80%, and 100% of their expected spend, so usage is visible before it becomes a problem. A dashboard breaks sessions into 16 anti-pattern categories so engineers can see what drove the cost. (Inside Uber’s software factory)
3. Finding-to-fix rate measures how often an AI reviewer’s output leads to a code change. Acceptance rate only shows what engineers clicked; finding-to-fix rate shows what held up. DoorDash’s AI reviewer converts 60.2% of its high- and critical-severity findings into a code change, up from 46% with the tool it replaced. Its DashBench benchmark tests this against 105 real cases, including clean PRs and PRs that were later reverted. (Inside DoorDash’s software factory)
4. Auto-approval rate measures the share of PRs a risk gate approves without human review. It captures quality at the checkpoint rather than inside the agent. Zalando’s approval bot scores every PR against rules drawn from past production incidents; about a third now pass automatically, across 250+ teams who use a variety of AI tooll. (Inside Zalando’s software factory)

What the Numbers Do Not Show

Where the dashboards fall short

Agentic work can accumulate hidden technical debt that most dashboards aren’t designed to show. Acceptance rate is one example: it counts correct findings, but it can never show the bugs an agent didn’t flag. It also can’t explain the findings that go unused. About 40% of DoorDash’s settled high- and critical-severity findings don’t lead to a code change, and no source says how many were wrong versus deprioritized. Volume hides outcomes too: DoorDash’s agent platform reports 130,000 tasks a month but no failure, rollback, or rejection rate outside code review. Even basic definitions are missing. Uber never defines “agent-authored” once a human edits the code, never says whether an agent PR can be merged without human approval, and describes no agent-caused incident. 

How teams fill the gap

The most useful benchmarks are built from what the agents actually did. DoorDash built DashBench from past PRs, and Uber chooses models with an internal benchmark that weighs cost per completed task, output quality, and reliability. Uber also collects ratings within the workflow: its reviewer asks developers to mark each comment as useful or not, and uses those ratings to tune the reviewer. Its test-generation team publishes its limits openly, stating that coverage isn’t effectiveness and reporting per-language success rates from about 20% to 80%. DoorDash is going further, building a harness that tests every reviewer change against a growing set of past incidents.

How to Start Measuring Your Own AI Software Factory

The companies in the index didn’t start with a full dashboard. They started with one narrow job they could measure, the way DoorDash chose code review, and recorded a before number to beat. From there, split spend into parts so you can track adoption separately from the work agents do per request; that’s how Uber’s cost per session fell while usage grew. Pair every volume measure, such as the share of PRs agents write, with a trust measure, such as the share of agent findings people act on. Decide what “agent-authored PR” means before the number reaches a board deck. Uber reports 11% of PRs when counting only those an agent wrote on its own, and more than 70% when counting any PR an agent touched. Benchmark against your own work, since Uber says its cost numbers won’t transfer to other companies; only its method will. And retire metrics that do harm, as Uber did with developer years saved.

All of this is easier when agent activity, delivery data, and spend live in one place. Port’s engineering intelligence view puts agent metrics next to the outcomes they’re judged against. See how SPS Commerce built a software factory on Port.

FAQ

Which metrics should a team track in the first month?

Pick one narrow agent job and measure three things: its volume, how often people act on its output, and its cost and time per task. Add adoption, counted as users and sessions, so you can see whether use is growing. DoorDash reports this mix for its reviewer agent: about $3 and seven minutes per review, along with the share of findings that lead to a code change.

How long does it take for AI impact to show up in engineering metrics?

Plan on a quarter or two, not weeks. DoorDash launched Flux in Q1 2026 and published reviewer results in May and benchmark results in July. Uber’s published cost trends span February to August 2026. 

Do DORA metrics still work once agents are writing most of the code?

Yes. A deployment is a deployment, regardless of who wrote the code. But DORA metrics can’t tell you which changes were made by agents. Tag agent-touched changes with a clear definition, then compare them with the rest. Uber’s 11% and 70% figures show why the definition matters.

How do you compare teams that use different AI tools?

Measure at the shared layer, not inside each tool. Uber runs one spend tier across all interactive tools and routes models and tools through central gateways. Zalando mandates no tool and measures at the pull request, through its risk classifier. DoorDash runs several agent harnesses on one platform.

Who should own AI impact reporting?

Whoever owns the shared platform. At Uber, a Distinguished Engineer led the cost work. At DoorDash, the AI reviewer came from the infrastructure engineering team, and a largely separate group built the benchmark that checks it.