Our AI pilots stall — and we are not sure why
AI pilots often stall for a reason that has little to do with the AI: the process the tool was meant to support was never captured. The tool meets undocumented exceptions, unclear ownership and rules that live in people's heads — and gives answers that sound right and are not. The fix is rarely a better tool. It is making the process ready: known, owned, and explicit about its rules and exceptions.
Do you recognise these signs?
- the demo looked impressive, but real cases disappoint;
- errors happen on exactly the cases your experts handle;
- people double-check every answer, so the time saving disappears;
- nobody can say which rule the AI should have followed;
- the pilot has no owner on the business side.
Why does it happen?
A general AI model knows a lot about the world and nothing about your company. Without your rules, exceptions and history it fills the gaps with what is usual — see why AI needs your business context.
Pilots also start without a baseline and without an agreed boundary between what AI may do and what stays with people, so nobody can tell whether it helped, or where it should stop.
What does it cost?
The direct cost is the budget and time of the pilot. The larger cost is trust: after one stalled pilot, “AI doesn't work for us” becomes the conclusion, and good opportunities are not tried again.
Meanwhile the experts are pulled in to check the AI's answers — the same people whose time the pilot was supposed to save.
The first step, this month
Take a handful of answers the pilot got wrong. Ask the expert who would have answered correctly what they knew that the tool did not, and write it down.
You now have the first list of knowledge to capture — and a much clearer view of whether the problem is the tool or the process.
How MentX helps
MentX captures how your company really works: the knowledge of your key people and the processes they run. We offer both a hands-on service approach — mapping your processes and capturing the knowledge behind them, from meetings, workshops and a guided intake — and our own personality-aware knowledge base, with full source attribution, in your own Knowledge Space. We build on the tools you already use and add best-practice solutions where they fit.
We make the process ready first: capture your operations — owners, decision rules, exceptions and data, with their sources — and then put AI to work where it helps, with operational excellence. The AI-ready process checklist shows what to check; ask us for the full version through the button below.
A question we often hear
Should we stop our AI pilots until everything is documented?
No. Keep the pilot small and use it to find the gaps: every wrong answer points to knowledge that is missing. Capture that knowledge for the one process the pilot supports, set a baseline, and agree what the AI may do. Then test again — on the same process, with the same cases.
