The AI SDLC: Shipping Software With Agents in the Loop

AI SDLC: faster code is not faster delivery

Sixteen experienced developers expected AI tools to make them about 24 percent faster. Researchers at METR tracked them on 246 real tasks in code they knew well. With AI allowed, the tasks took 19 percent longer.

Afterward, the developers still believed they had been 20 percent faster. Felt faster. Was slower.

Feeling fast is free. Measuring takes work. This guide shows you how to run an AI SDLC you can actually measure.

An AI SDLC is the software development lifecycle with AI agents doing work inside each step, while people own every gate.

The short version fits on a sticky note. Small changes, tests first, a person on every merge, and one team at a time.

What is an AI SDLC?

An AI SDLC is the normal path a code change walks, with agents doing more of the walking. The software development lifecycle itself is old. Code gets written in development, tested, reviewed in staging, then released to production.

Staging is a near-exact copy of the live system, so tests there mean something. Rejected work moves back a step, and accepted work moves forward. Jack Stanley and Erik Gross lay out this sequence plainly in The Project Management Handbook.

Agents change who does the work inside each step. They draft a plan from a ticket, write code against tests, and summarize what changed.

They do not change who signs off between steps. People own the gates.

Think of an agent as a very fast new hire. Nobody lets a new hire ship to customers on day one.

For the buying and setup side, see our guide to AI coding agents for enterprise teams.

What did the METR and DORA research find?

Two research groups looked hard at AI and software delivery. Their findings point the same way.

METR ran a randomized trial on real work, so its result carries weight. Still, METR now marks those early results as out of date. Its follow-up update says developers are likely faster now, but that the new data is weak.

So the 19 percent is not the lesson. The lesson is that people misjudged their own speed by 39 points: 20 percent faster in their heads, 19 percent slower on the clock.

DORA is Google's long-running research program on software delivery. Its 2024 report found AI adoption raised individual productivity, flow and job satisfaction.

Delivery told a different story. A 25 percent rise in AI adoption went with an estimated 1.5 percent drop in delivery throughput and a 7.2 percent drop in delivery stability.

The report's advice sounds boring. Small batch sizes and robust testing still decide results. Boring works.

Here is our reading. Agents make code cheap, so changes get bigger, and big changes get skimmed instead of reviewed.

That is not a tool problem. It is a batch-size problem.

Measure four delivery numbers before you add an agent

Before any agent touches your code, write down four numbers. They come from Accelerate, the research book from Nicole Forsgren, Gene Kim and Jez Humble.

Lead time is how long a change takes from commit to running in production. Deployment frequency is how often you ship. Time to restore is how fast service comes back after an incident.

Change fail rate is the share of releases that need a hotfix, a rollback or a patch. The first two measure speed. The last two measure stability.

Do you have to trade one for the other? No.

In the 2017 survey data, high performers deployed 46 times more often than low performers. Their lead time was 440 times faster, they recovered 170 times faster, and their changes failed one fifth as often.

Speed and stability rose together. The teams that built quality in got both.

Chip Huyen applies the fail rate to AI systems in AI Engineering. Her rule: if you cannot say what share of your changes fail, make your system easier to see into first.

No baseline, no result. Just a story.

Keep every agent change small enough to merge in a day

Small batches are the cheapest safety tool in software. The Accelerate research says exactly how small.

Teams that did well kept fewer than three active branches open at once. Each branch lived less than a day before it merged. They never froze code to stabilize it.

A pull request is a proposed change that waits for review before it merges. Agents push against every one of those limits, because writing code costs them almost nothing.

So they happily open a pull request that touches twenty files. That's not a pull request. That's homework for whoever reviews it.

Our rule: one task card per agent branch, and the branch merges or closes within a day. If the change will not fit in one sitting of review, split the card.

The same goes for the task itself. If you cannot size it, break it down before anyone starts, human or agent.

Write the test before the agent writes the code

Tests written first check the code. Tests written after just agree with it.

That is the idea behind test-driven development. AI Engineering applies it to AI work and calls it evaluation-driven development. Decide how you will grade the work before you build it.

The team behind one of the first big AI coding assistants worked this way. In Prompt Engineering for LLMs, John Berryman and Albert Ziegler write that their oldest code was the evaluation, and that it let them move fast.

Their test is worth copying. Take code with passing tests, delete one function body, let the AI rewrite it, and run the tests again.

Who owns the tests matters too. In the Accelerate data, tests that developers wrote and maintained predicted performance, and tests owned by QA or an outside firm did not.

So the agent may draft tests, but a developer owns them. Unreliable tests go into a separate quarantine suite or get deleted, because a test nobody trusts is noise.

Give every agent task the same card. A user story names who wants what and why, plus one measurable line that defines done.

Put the card inside the agent's job spec, and grade the output the way you would evaluate any AI agent.

A person reviews every change, and the agent never merges its own

Who should approve agent changes? A peer, not a committee.

Accelerate compared four approval setups. Approval by an outside body, like a manager or a change board, went with slower delivery and showed no link to fewer failures.

Teams using peer review did better, and the authors pair it with an automated pipeline that rejects bad changes. They call outside sign-off a form of risk management theater.

We turn that into three rules for agents. First, someone who did not write or prompt the change runs the code review and records the approval.

Second, changes reach production only through the automated pipeline. Third, agents get least privilege: no production credentials, no secrets in prompts, no power to merge.

Do not rely on a polite instruction to keep an agent in line. Berryman and Ziegler warn that a model told to double-check first will still skip it some of the time.

A prompt is a request. Code is a lock.

Block risky actions in code, and make a person sign off. Write the full list into your AI governance framework.

Prove, Transfer, Scale: roll out one team at a time

The Prove-Transfer-Scale method is our rollout order for AI coding agents. Each stage answers one question before the next one starts.

Prove. Pick one team and one kind of work. Record the four numbers for four weeks before agents, then for four weeks with them.

Transfer. Write the playbook: the task card, the review checklist, the permissions, and a log of what the agent got wrong. A second team then runs it, with the first team helping.

Help one team at a time. Matthew Skelton and Manuel Pais set that as a firm rule in Team Topologies: close collaboration with at most one other team at once.

Scale. Roll out wider only after two teams moved the four numbers the same way. Later teams use the playbook without hand-holding, under the same guardrails.

Expect a dip in the middle. In the Accelerate data, middle-tier teams spent more time on rework than the lowest tier.

How to roll out AI coding agents badly in four easy steps:

1. Buy seats for the whole company on Monday.

2. Count logins and call it adoption.

3. Let agents merge their own work to save time.

4. Learn what broke from your customers, and hope like hell nobody asks for the numbers.

Skip a stage and you scale a feeling, not a result.

Which work should AI agents take first?

Start where mistakes are easy to spot and cheap to undo. Move to harder work only after the team knows how the agent fails.

The research backs that order. Huyen cites McKinsey data showing about twice the productivity on documentation and 25 to 50 percent more on writing and refactoring code, with minimal gains on highly complex tasks.

She also explains why coding took off first. You can run code and check it.

Work you can verify is work you can hand over. Work you cannot verify stays with a person.

Good first work: tests for existing code, small bug fixes, documentation, routine updates. Poor first work: security code, billing logic, and large changes across many parts of the system.

The log of agent mistakes from this stage is the most valuable page in your playbook. Not the prompts. The mistakes.

Easy to check first. Hard to check later.

Your developers say they are faster. Check the four numbers

They may be right. They may also be standing in the gap METR measured.

Believe them? Doubt them?

Neither. Measure.

Ask about behavior, not labels. The Accelerate team never asked whether people practiced continuous integration, because everyone says yes. They asked whether tests run when code is checked in.

Then read the four numbers together. Faster lead time with a flat change fail rate is a win you can transfer.

Faster lead time with a rising fail rate means the batches grew or the review got lighter. Faster to write. Slower to trust.

Speed has a price tag too. For the cost side, see controlling AI token spend.

Run it with AI this week

Paste the prompt below into your preferred AI. It asks about your team, then drafts your baseline, your agent rules and your Prove plan.

It will not invent your numbers. Those come from your version control and your incident log.

1. Pull the four numbers for the last four weeks, before any new agent work starts.

2. Pick one team and one kind of work that is easy to check and cheap to undo.

3. Write the first five task cards, each with its test, and turn on required review for agent branches.

Rules in place? Next, wire them into your agent harness so they run on every change. Or have us build the system with the gates already in place.

Remember the developers who felt 20 percent faster? Let the four numbers drive, and let the feeling ride in the back seat.