Why AI pilots stall at the pilot stage

There is a conversation I have had in different organisations in almost identical words. The pilot is finished, the results have been presented, everyone was enthusiastic. Six months later someone on the board asks how it is going, and it turns out three people use it occasionally.

The first reaction is usually that the technology fell short. The model was not good enough, the connection to the source system was missing, the interface was awkward. Sometimes that is true. But in most of the cases I have watched closely, the technology simply worked.

What actually goes wrong

A pilot proves that something is possible. It does not prove that anyone will do it. Those are two different questions, and the second is rarely asked before the building starts.

Anyone adopting a new application has to do something differently from yesterday. Skip a step they have taken for years. Let go of a judgement they were used to making themselves. Prepare for a meeting in another way. Small things, but they are tied to how the work is organised, to what people are held accountable for, and to what happens when something goes wrong.

As long as nobody has worked out how that job actually runs — where decisions are made, which steps people skip, what gets agreed in the team meeting — an application is an offer nobody asked for.

None of this is new

I spent eight years on data foundations: data warehouses, reporting environments, governance frameworks. Across those years I kept seeing the same pattern. The dashboard was excellent. The figures were right. And the decision was still made on instinct.

The technology was never the problem. The problem was that nobody had worked out how decisions were being made.

With AI the same thing is happening now, one technology generation later. The instruments are more powerful and the promises larger, but the gap sits in the same place: between what a system shows and what someone dares to do with it.

What adoption programmes measure

Vendors offer adoption programmes alongside their licences. Those measure activation: how many people logged in, how many sessions a week, how many prompts. Tidy numbers, and they usually rise tidily.

They say nothing about the question that matters. Whether a decision comes faster. Whether a process has fewer steps. Whether the quality of the work differs from last quarter. Someone who logs in and then does exactly what they always did counts as an activated user.

Reversing the order

The practical consequence is that adoption is not a closing stage but a starting point. Not a training module after delivery, but the research that determines what should be delivered at all.

In concrete terms, that means work happens before anything is built:

• Establish how the work runs now, alongside the people who do it.
• Record where it stands today, so there is something to compare against later.
• Determine who will use the application and what it takes for them to keep using it.
• Only then decide what gets built.

That feels slow. In practice it rarely is: the research takes weeks, a pilot nobody uses takes a year and a budget.

The question that counts

With every AI investment there is one question that settles the matter, and it is not whether the technology works. It is whether the work changed. If nobody recorded that beforehand, the question cannot be answered afterwards — and it stays a pilot that did the job.