Estimating AI Prototypes in Two Stages: From Zero to Demo, From Demo to Production

When I build a prototype with AI, the first half goes almost suspiciously fast. In about a week I have a clickable prototype, complete with a concept mock-up and responsive layout. On a more complex project the AI bill is around US$200, not counting my own time. Getting it ready to go live takes longer: permissions, concurrency, charge and refund handling, testing and observability all need work. AI helps with those too, though the cost hasn’t fallen nearly as much.

I estimate and quote for “zero to demo” and “demo to production” separately. The first stage follows the actual pace of my AI workflow; the second is based on past production work, with verification tasks listed individually. No AI discount on that part.

1. The first half: one delivery a week

My current prototyping workflow looks roughly like this:

Claude Design   concept mock-up, to agree direction and visuals first
Google Stitch   responsive layout
SvelteKit       clickable prototype
Claude Code     convert the prototype to Nuxt
my own review   read it through before delivery

I prototype in SvelteKit mainly because Svelte 5’s runes make state and derived values explicit. The roles of $state, $derived and $effect are clear at a glance. In my experience, AI produces more stable code with them, and there’s much less guessing about whether changing a variable will trigger an update. I wrote about that in Svelte 5 Runes: Reactivity Out in the Open.

Once the prototype is agreed, Claude Code converts it to Nuxt. Moving between frameworks used to be close to a rewrite; now it fits into the same week.

By then the screens and user flows are there, with fake data to run through the main happy path. The client can try it and ask for changes, which pins down requirements far better than a spec. At about one delivery a week and around US$200 in AI costs for a complex project, this stage has become much cheaper.

It’s easy to assume the rest of the project has become cheaper by the same proportion. With complete screens you can click through, it looks 80% done, yet most of the work needed to go live hasn’t started.

2. The second half: the cost is in verification

I built a minimal game provider system prototype a while ago. Once the features were in and the flows ran, I read through it myself and fixed six High-severity problems. They fell into permission checks, state isolation between requests, charge and refund handling under concurrency, amount validation, and the way sensitive values were compared.

None of the six would show up in a demo. With one request at a time, sensible amounts and the right permissions, the happy path looks fine on screen.

After those fixes I ran a round of multi-agent review. The 46 agents produced 16 findings; a second group tried to refute each one, and 4 survived. I fixed all of them. Cutting the list from 16 items to 4 left far fewer for a person to assess individually, though the 12 rejected findings still need spot checks to confirm they really don’t hold. I’ve covered the cost-benefit of that verification layer and its risk of missed problems in Counting the Cost of AI Code Review: Bugs Found, False Alarms and People’s Time, so I won’t repeat it here.

Some charge and refund problems only turned up in end-to-end and integration tests. I used a real PostgreSQL database and a fake counterparty server, then injected faults to make the counterparty time out, return errors or send duplicate responses. That let me check whether charges, refunds and reconciliation still came out right.

The fault scenarios looked something like this. When both the payout callback and the status-query callback timed out, the round was left pending, and a background worker later resent it with the same transaction id; however many times it was resent, it had to settle into a single transaction. When I killed the service halfway through an e2e run, it had to pick up from where it stopped once it came back. If the cache connection dropped, the kill switch had to fail closed rather than let traffic through because it couldn’t read its setting.

The e2e tests can mislead you as well. Requests sent within the same second with the same body and path ran into the replay-protection nonce check, which looked like a bug in the code when the fault was in how the test was written. The demo slot game’s results were random, so the e2e suite also failed now and then, until I switched the test environment to replayable seeds. I described how in Unpredictable Game RNG in Production, Replayable in Tests: Separating the Two in Go.

Unit-test mocks can simulate timeouts and errors too. Integration tests go through the database transactions and network calls, so transaction isolation, locking and retry timing are all real. Reviewers reason about the source code; these tests run the system. They tend to uncover different problems.

On that project, most of the cost after the prototype came from:

Verification through manual and multi-agent review, end-to-end tests and integration tests. Someone has to judge the results at every layer.
Concurrency, including race conditions, along with retries and state transitions that a demo sending one request at a time rarely sets out to exercise.
Payment handling: limits, idempotency, refunds and reconciliation. One mistake costs far more than writing a few more tests.
Observability through logs, metrics and traces, so you can identify the transaction and the step when something goes wrong.
Data volume and architectural semantics. A full table scan goes unnoticed among a few dozen rows of fake data, and modules AI generates in separate batches don’t necessarily agree on naming or meaning.

AI writes plenty of the review comments and tests, and runs the tests as well. A person still has to decide whether each finding holds and whether a failing test points to a bug in the code or in the test. AI hasn’t saved much time on those decisions.

3. AI tends to add more code

Ask AI to fix a problem and it more often adds another condition or helper, so the wrappers pile up; it seldom deletes anything or merges duplicated logic without being asked. An LLM can produce code quickly without much discipline about abstraction, which adds to the cost of the second half.

During prototyping that hardly matters, as long as it works. Once you’re heading for production, every bit of new code needs tests and monitoring after release. An AI coding workflow needs checks for code that can be removed or simplified. Put a refactor gate before a feature is merged, checking for code that can be deleted or merged. AI also readily writes near-identical functions in different files, so run regular duplication checks and merge them.

A complexity budget gives modules or functions a ceiling. Once they exceed it, refactor before adding more features.

My own order of priorities is Security > Simplicity > Correctness > Performance. I prefer to meet a requirement with less code, deleting what I can first. As I wrote in “It Builds” Is Not the Same as Well-Designed, code that runs and passes its tests tells you only that those tests found no failures. You still have to check whether the design makes sense. AI has made “it runs” very cheap, which makes that gap easier to overlook.

4. Feeling faster is unreliable

Feeling faster with AI is a poor basis for an estimate. It may not match how much faster you actually are.

In July 2025 METR published a randomised controlled trial with 16 experienced open-source developers, who completed 246 tasks on mature projects they had worked on for five years on average. Before the trial they expected AI to cut completion time by 24%; afterwards they reckoned it had saved them 20%. The measured result was a 19% increase in completion time when AI was allowed.

On 24 February 2026 METR published a follow-up experiment with 57 developers and more than 800 tasks. It reported a “speedup of -18%” for the original participants, with a confidence interval from -38% to +9%, and -4% for newly recruited participants, with an interval from -15% to +9%. Both intervals cross zero. The post doesn’t separately define the sign convention. The earlier study used positive numbers for extra time taken; following that convention, a negative value means less time. METR’s own reading is that there are some signs of speedup, but the evidence is very weak.

METR also calls the data an “unreliable signal”. One reason is that 30% to 50% of developers said they chose not to submit some tasks because they didn’t want to do them without AI. METR considers its estimate more likely a lower bound on the true effect. The original study’s page now carries a notice at the top saying “These results are out of date”, with a link to the update.

Selection bias affects the follow-up, so it can’t yet provide a reliable estimate of the speedup. In the first experiment, the developers’ own estimate of the change in completion time was nearly 40 percentage points away from the measured one. Work on mature projects resembles the second half of a prototype: changes have to fit the existing architecture and codebase and pass the tests. Estimating that work from how fast the first half felt is riskier still.

5. Quote in two stages

This is how I separate the two stages in a quote. The table is illustrative:

Stage	Output	Main cost	How to estimate
Zero to demo	Concept mock-up, responsive layout, clickable prototype	AI spend, requirements back-and-forth	At the AI workflow’s actual pace
Demo to production	Permissions, concurrency, payment handling, tests, observability	Human judgement and verification time	From past experience of going live, with no AI discount

Itemise the second stage so the client understands that a clickable prototype isn’t close to finished. A client may only want the first stage, for investors or an internal proposal. In that case, quote for it alone and state clearly that the system isn’t ready to go live. Estimate the second stage separately if they later want to put it into production.

Splitting work between models can save some of the AI spend. In September this year the Spotify engineering blog described putting a smaller model in front of Claude Code to handle bulk file reading. The author ran four scenarios on a Java monorepo, comparing the tokens Claude used when it read files or wrote code itself with what it used when it read the smaller model’s summaries or handed the writing over. Bulk reads used around 90% fewer tokens on average. Those are Claude-side tokens; the post doesn’t account for the smaller model’s own usage.

The same post says reasoning and editing couldn’t be handed to the smaller model; in the author’s testing it missed a thread-safety bug that Claude caught. The saving covers file-reading tokens. Reasoning and editing still need a model capable of handling them reliably, followed by human review.

Notes

With AI, “zero to demo” takes about one round a week and around US$200 of AI spend on a complex project. That stage has become much cheaper.
Verification, concurrency, payment handling, observability and architectural semantics account for the cost after the prototype. Not one of the six High-severity problems was visible in a demo.
Multi-agent review narrowed 16 findings to 4 that held. Some of the charge and refund problems were only found by tests with a real database, a fake counterparty server and fault injection.
AI tends to add code. A refactor gate, duplication checks and a complexity budget give you opportunities to remove or simplify it.
METR’s first experiment showed perceived and measured speed can differ widely, and the follow-up can’t yet estimate the speedup reliably. Don’t estimate the second half from the pace of the first.
Quote for the two stages separately, basing the second on past production work and listing the verification tasks.