Pilot Purgatory: why AI projects don't ship

Pilot Purgatory is where AI projects go when they demo brilliantly and never ship. The proof of concept works, everyone in the room is impressed, and then months later nothing has actually changed in the business. The pilot is still a pilot. It is the most common and most expensive way for AI money to disappear, and it is almost never a technology problem.

I see it constantly. A business gets excited about AI, runs a proof of concept, and the demo goes well. There is a real sense that something has been unlocked. Then the momentum quietly drains away. The pilot never quite makes it into daily use, the enthusiasm fades, and a year later the honest summary is that a lot was spent and very little changed. If that sounds familiar, you have been in Pilot Purgatory, and you are in good company.

What Pilot Purgatory actually is

A pilot is a demonstration of the easy part. It proves an idea can work in principle: on clean data, in a controlled setting, with the person who built it standing right next to it. Production is a different thing entirely. It has to cope with messy real inputs, the awkward edge cases, actual users who did not build it, and the systems you already run. It has to keep working when nobody is watching and nobody is available to nudge it.

Pilot Purgatory is the gap between those two states. Businesses get stuck in it because the pilot was so convincing that everyone assumed the hard part was done. It was not. The demo is the first twenty per cent. The last mile from demo to dependable, used system is the other eighty, and it is a genuinely different kind of work.

The demo is not the project. The demo is the first twenty per cent of the project, dressed up to look like the finish.

How projects end up here

There are three reasons, and they are mostly about who owns that last mile.

The demo rewards the wrong thing. A pilot is built to impress. It picks the cleanest example, quietly avoids the edge cases, and skips the integration work. That is fine as a test of the idea, but it sets a false expectation that production is just around the corner. Getting from a good demo to a system people can depend on is where the real engineering and, more importantly, the real judgement lives. None of that shows up in the demo, so none of it gets planned for.

No one senior owns the last mile. This is the big one. In most arrangements, the senior person who understood the problem and sold the vision steps back once the pilot lands, and the unglamorous work of hardening it, integrating it and getting people to actually use it is handed to whoever happens to be free. That loss of quality has a name: I call it the Handoff Tax. The last mile is exactly where experience matters most, and it is exactly the point at which experience tends to leave the room.

"Done" was never defined as "live". If success was quietly defined as a working demo, then the project succeeds the moment the demo works, and everything after that starts to feel like scope creep. If success is defined as "live in the business and being used every day", the demo is simply the start. Most pilots stall because nobody agreed, at the beginning, which of those two "done" meant.

How to tell you are in it

The signs are consistent. The pilot has been "nearly ready" for months. Every conversation is about adding a new feature rather than getting the current one into real use. The people who built it can make it work, but nobody else can. It runs on someone's laptop or a test account rather than inside the business. And the honest usage number, outside of demos, is close to zero. If more than one of those is true, the project is not progressing. It is parked.

The last mile is a seniority problem, not a resourcing one

The instinct, when a pilot stalls, is to throw more hands at it. That usually makes things worse, because the last mile is not about hands. It is about judgement: which edge cases genuinely matter and which can wait, where to put a human check, what to simplify, when something is good enough to ship and when it is not. Those are all senior decisions. Add a few junior people to a last-mile problem and every one of those decisions now has to be escalated, so the whole thing slows down rather than speeds up.

This is the reason my own model is one senior person carrying a project from first scope to live, using AI to cover the ground a whole team used to cover. I call it AI-Leveraged Delivery, and the entire point of it is to keep the same judgement on the project at the end that it had at the start. The last mile gets the senior attention it needs, because there is no one else to hand it to.

What it looks like in practice

Take a common one. A business builds a tool that drafts replies to customer enquiries. In the demo it is genuinely impressive: you paste in an enquiry, out comes a clean, on-brand answer. Everyone agrees it will save hours. Then it meets reality. Real enquiries arrive as forwarded email chains with three questions buried in them, half of them reference an order the model cannot see, and a handful each week are complaints that need a careful human touch. The tool handles the tidy cases and quietly mangles the messy ones, so the team stops trusting it and goes back to doing it all by hand. The pilot was not wrong. It just never crossed the last mile.

Getting it out is not glamorous work, but it is decisive. It means connecting the tool to the order system so it has the context, drawing a clear line so that anything resembling a complaint is routed straight to a person, adding a light review step for the rest, and then sitting with the team while they use it for real and fixing the ten small things that make the difference between "clever" and "relied on". None of that is in the demo. All of it is what makes the demo worth having built. That is the last mile, and it needs someone senior enough to make those judgement calls quickly.

How to get out of Pilot Purgatory, or avoid it

Four things get a project across the last mile. None of them is technical.

Scope to production from day one. Decide before you start what "live and used" actually looks like, and treat the pilot as one step towards it rather than the destination. A pilot with a defined route to production is useful. A pilot for its own sake is where the money goes to die.

Put one senior person on the whole thing. The person who scopes it should build it and ship it. No handoff, no tax. Continuity of judgement from start to finish is worth more than any amount of added capacity.

Fix the price and the date. A fixed price and a real delivery date force the last mile into the plan, instead of leaving it as open-ended "optimisation" that never quite finishes. Constraints are what turn a pilot into a shipped thing.

Define done as used, not demoed. The project is finished when people in the business are using it without you standing there, not when it works once in a meeting. Hold that line and most of the traps that lead to Pilot Purgatory simply close.

The honest summary

Most AI that fails does not fail because the technology could not do the job. It fails because nobody owned the boring, difficult, decisive last mile between a demo and a working part of the business. Pilot Purgatory is the result, and the way out is not a cleverer model or a bigger team. It is a single senior person who treats "live in your business" as the only definition of done. If you have a pilot that has been nearly ready for a while, that is the conversation worth having.