Self-Improving AI Software Factory: How Feedback Loops Make Agents Better Over Time

Key Takeaways

• A factory that only runs tasks ships the same quality in month six as in week one, because nothing turns a finished run into a change.
• A self improving AI factory closes that gap with scorers that grade completed runs, a step that finds the pattern across a batch of grades, and a write-back into the skill, prompt, or golden path the next run reads.
• Score what you can write a rubric for and track it on a scorecard the entity’s owner answers for: task compliance, procedure compliance, token cost, code quality, and defects the change caused downstream.
• Keep a person on the write-back, because an edit to a shared skill changes how every team that installs it works.
• Read improvement as a trend over weeks: the score line, autonomy share, change failure rate, and cost per unit of work shipped.

Why Software Factories Stop Improving After Launch

Your triage agent will misroute the same class of ticket in month six that it misrouted in week one. The model underneath it improves when a vendor ships a new one, and the factory around it sits where you left it, because nothing in the line turns yesterday’s bad run into today’s fix.

That is the gap between automation and improvement. An AI software factory moves triage, spec, implementation, review, and verification off a person’s plate, which is the automation half. Running the work is not grading it, and an agent nobody grades reads the same skill file and the same prompt on run 900 that it reads on run one.

Human teams close this loop without naming it. A reviewer tells an engineer once that the codebase names global variables a particular way, and every pull request after that follows the convention. Agents drop that correction when the session ends, so the same mistake comes back on the next run and the one after that. Most teams patch this by hand today, collecting complaints about agent output in a Slack channel and editing prompts when someone gets around to it.

The raw material for a better loop only exists if something captures it. Interactive agents on individual laptops leave nothing to grade, since the session closes and the trace goes with it. A factory writes every run to one place, with the diff, the tools called, the tokens spent, and whatever the human did next.

How a Feedback Loop Turns Agent Runs Into Fixes

An AI feedback loop runs in four moves, and the fourth is the one teams skip.

Capture the run. The trace lands with its outcome attached: tests passed or failed, reviewer comments, the edits a human made to the diff, whether the ticket reopened a week later.

Grade it. A scorer pairs a run with a rubric, and assigns a grade, from code, from a person, or from another model acting as judge. Run scorers on a cadence, every few hours against a sample, rather than on each run as it completes. Grade the dimensions that map to real failure: whether the run did the task, followed the procedure, held up in review, and what it cost against the work it shipped. Behind the scorer sits a scorecard on the agent’s catalog entity: the standard that says what good looks like, with tiered levels the way a service scorecard sets production readiness, so every run is graded against a bar the agent’s owner set rather than an ad-hoc call.

Find the pattern. Reading one grade tells you little. Reading a batch, say 20 runs of the triage agent, shows you what the low scorers got wrong and what the high scorers did right. The output is a pattern rather than an average, because “routes database alerts to the platform team when the service owner is set” is actionable and “scored 3.8 on compliance” is not.

Write the fix back. The pattern becomes a small, specific edit to the skill or prompt the agent reads, and a person reviews that edit before it lands. A tight, explicit correction gives a strong signal even from a small sample, where a vague one teaches the agent nothing. A person writes that edit at first, and once the loop earns trust you can build an improver agent to propose it for review instead.

The last move is where the architecture matters. The signal comes from runtime, and the change lands at build time, in the skill, the prompt, the scorecard, the tool scope, or the model the golden path assigns. Treat those as code and the improvement stops being a tuning session someone ran once. Fix the skill and every team that installs it inherits the fix, which is the difference between one agent getting better and the factory getting better. The agentic SDLC puts the registry and the skills catalog at the center for this reason: they give the loop somewhere to write to.

Choosing What to Score in Agent Runs

Score what you can write a rubric for, and attach each score to something that has an owner.

Start with the dimensions that map to a real failure. Did the run do the task asked of it, did it follow the procedure, which for a migration agent means touching only the files in scope, what did it cost in tokens against the work it shipped, did the code hold up in review, and did the change cause a defect two weeks later. Scoring the last two means knowing what the run touched and what depended on it, which is what the Context Lake, the factory’s knowledge graph, holds. AI agent evaluation grades runs against your workflows rather than a public benchmark, since nobody ships against SWE-bench.

Two kinds of measurement answer different questions. Sampling live runs tells you whether output is drifting. A benchmark replays a fixed task set, built from your own past runs, to compare one model, harness, or skill version against another. Run the first continuously and the second when you change something big.

Set the sampling rate deliberately, since scoring every run buys complete coverage at full price and scoring a sample buys a cheaper, noisier signal.

Watch the rubric itself for rigidity. A scorer tuned too tight starts blocking valid changes, and a scorer that blocks good work costs more than the bad work it catches. Grade the scorer on the same evidence you grade the agents on, and loosen it as the agents earn trust.

Last, point every score at a catalog entity: this agent, this skill, this MCP server, each with a named owner. A score that lands in a dashboard belongs to nobody. A scorecard sets the standard on the entity, with tiered levels an owner answers for, so the grade measures the agent against a bar someone set and a review can track that standing against it over time.

Where Humans Stay in the Loop

Human in the loop AI in a self-improving factory means two gates, not one.

The first gate sits on the work. The line scores risk and routes judgement calls to a person, so a low-risk dependency bump merges on its own and a change to a critical service waits for the on-call engineer to approve it in Slack.

The second gate sits on the improvement itself, and it is the one teams forget to build. An agent proposing an edit to a shared skill is proposing a change to how every team using that skill works, so review it like the pull request it is.

What people feed the loop matters more than what they approve. A thumbs up on a review comment tells it almost nothing, while telling the agent that it suggested renaming a variable the codebase has a convention for gives the improver agent something to write down. Capture the reason rather than the verdict, inside the tool people already work in.

Keep a person on three changes permanently: any edit to a skill that multiple teams install, any change that widens an agent’s permissions or tool access, and any promotion of an agent from propose to act. Autonomous software development works as a setting you raise per stage once the evidence supports it, not as a switch you flip for the whole line.

Signs the Factory Is Improving Over Time

• The score line moves on a fixed rubric. Hold the rubric steady for weeks and watch the trend, because changing both the rubric and the score measures nothing.
• Repeat failures drop. Count how often a pattern you already fixed shows up again. This is the one metric that grades the loop rather than the agents.
• Autonomy share rises while change failure rate holds. Track the share of tickets an agent closes end to end and the share of pull requests merged without a human edit against your change failure rate. If both climb together, the loop is optimizing for throughput and charging you in defects.
• Cost per unit of work shipped falls. Token spend and percentage of code written by AI rise while delivery stays flat. Cost per merged pull request or per resolved incident ties spend to output.
• Review load per change drops. Fewer runs reaching a person for the same shipped volume means the gates catch less, because less reaches them wrong.

Pick one stage where output is verifiable and mistakes are cheap to undo, usually triage or review. Write one scorer for it, run it on a sample for two weeks, and let a person apply the first three fixes by hand before an improver agent touches anything. The teams pulling ahead are not running better models than you. They capture their runs, grade them against their own standards, and write the answers back into the skills their agents read tomorrow.

FAQ

Is self-improving AI the same as recursive self-improvement?

No. Recursive self-improvement describes a model rewriting its own weights or architecture. A self-improving factory leaves the model alone and edits the scaffolding around it: skills, prompts, context, tool scope, and routing. People approve each change, and every edit stays versioned and reversible.

Does self-improvement increase AI token costs?

Scoring runs and running improver agents both consume tokens, so the loop carries a real cost. Sampling instead of grading every run keeps it small, and the loop pays for itself by cutting retries, bad routing, and oversized context. Measure cost per unit of work shipped, not total spend.

Can self-improving agents make the factory worse?

Yes, if the loop writes unreviewed changes. A rigid scorer teaches agents to chase the rubric instead of the outcome, and an unreviewed skill edit can spread a bad habit to every team that installs it. Gate skill edits behind human review, version every change, and track change failure rate beside the score.

Which SDLC stages benefit most from self-improvement?

Stages with a clear verification signal improve fastest: triage, code review, test generation, and incident classification. Each produces a checkable outcome, so a scorer grades it without ambiguity. Open-ended design work gives the loop weaker signal, so start where correctness is easy to judge.

Do self-improving factories work with any coding agent?

Yes, when the factory layer stays separate from the agent. Claude Code, Codex, Copilot, and in-house agents all emit runs you can capture, score, and feed back. The loop lives in the platform holding the traces, the skills, and the registry, so you swap agents without rebuilding it.

{{from_manual_to_autonomous_engineering}}