A demo is a controlled test with a friendly operator, a clean input and no consequences. Production is none of those things. The agents that disappoint people almost never fail because the model could not do the task — they fail because the surrounding system assumed conditions that stopped holding the moment real work arrived.
These are the five failures that account for most of it, in roughly the order they tend to appear.
1. It answers confidently from nothing
A language model asked a question it has no grounds to answer will usually answer anyway, fluently. In a demo this is invisible, because the questions were chosen by someone who knew what the agent could handle. In production it produces answers that are plausible, specific, wrong, and delivered in the same tone as the correct ones — which is precisely what makes them expensive. A visibly broken agent gets reported. A confidently wrong one gets believed.
What prevents it
Retrieval before generation, against your own material, so the agent is answering from your documentation rather than from general training. And an explicit path for "I do not have this" — an agent that is permitted to decline is far more useful than one that is not, because its answers mean something.
2. An integration fails silently
The agent calls your CRM. The CRM is down, or rate-limited, or has changed a field name. The call fails. If nobody designed for that, the work does not go anywhere — it does not queue, it does not escalate, it does not appear in a report. It is simply gone, and you find out when a customer asks why nobody got back to them.
This is the most common failure in automated workflows generally, and it has nothing to do with AI. It is worth saying plainly because it is also the most preventable: every automated step needs a defined destination for the cases it could not complete.
What prevents it
Failures route to a person by default. Not to a log file that nobody reads — to the queue the work would have gone to anyway if the agent had not existed. The worst acceptable outcome of an automation failing is that the process reverts to manual.
3. Nobody defined what it must not decide alone
Scope creep in an agent is not usually a feature request. It is the gradual discovery that the agent has been deciding things nobody meant it to decide — issuing a refund, making a commitment on a date, closing a ticket that needed a human read. Each individual case looks reasonable. The pattern is a system operating outside its mandate because the mandate was never written down.
What prevents it
The list of things that escalate is part of the specification, not an afterthought — and it is a business decision, not a technical one. It also has to be enforced by what the agent can actually reach: an agent with credentials for an operation will eventually perform it, so permissions should be scoped to the mandate rather than to convenience.
4. The process moved and the agent did not
This is the slow failure, and the one that catches organisations who did everything else right. The agent was built correctly against the process as it ran in March. By November the form has two new fields, the team has changed how they triage, and a supplier has started sending invoices in a different format. Nothing broke — the agent is simply handling a slightly wrong version of the job, a little more wrongly each month.
What prevents it
Scheduled review against what actually happened, rather than against the specification. Reading a sample of real cases every month is unglamorous and catches drift long before a metric does.
5. It was measured against nothing
A surprising number of automation projects cannot say whether they worked, because nobody recorded what the process cost before it was automated. Six months later the question "did this save us anything?" has no answer that is not an argument, and the project gets judged on impressions instead of evidence.
What prevents it
Measuring the process before touching it — volume, handling time, rework rate. That baseline takes little effort at the start and is impossible to reconstruct later.
The pattern behind all five
None of these is a modelling problem, and none of them is fixed by a better prompt. They are all the same mistake in different clothes: treating the agent as the deliverable, when the deliverable is a working process that happens to have an agent in it.
Which is also why an agent is not finished when it ships. The failure modes above are almost all discovered after deployment, by someone who is still paying attention.