Two numbers, from the same eighteen months.
Blended cost of AI inference: down 67%, from $18.40 to $6.07 per million tokens between Q1 2025 and Q1 2026. Total enterprise AI spend: up 483% from 2024 to 2026.
Prices fell by two thirds and bills rose fivefold. Anyone who lived through the first decade of cloud recognises the shape of this immediately, and knows how it ends: a CFO asks a question nobody can answer, and a promising programme spends a quarter defending itself instead of shipping.
Seventy-three percent of enterprise agentic AI projects overshoot their budget. Some by a factor of 2.4.
Why agents cost so much more than chatbots
This is the mechanism, and it is worth understanding precisely because every mitigation follows from it.
A chatbot interaction is one request and one response. You pay for the input, you pay for the output, and the transaction ends.
An agent run is a loop. It reads its instructions and context, decides on a tool call, executes it, reads the result, reasons about what to do next, and repeats until it thinks it is finished. Ten iterations is unremarkable. Fifty happens. And on most implementations, every single iteration resends the entire accumulated conversation, including every prior tool result.
So the token consumption of an agent run does not scale linearly with the work. It scales closer to quadratically with the number of steps, because step twenty is paying to re-read the output of steps one through nineteen.
Add a few common patterns and it compounds:
- Multi-agent architectures. A supervisor agent coordinating four specialists is running five loops, and passing context between them.
- Retrieval. Every retrieved document is input tokens, on every iteration that carries it.
- Reasoning models. Extended thinking produces tokens you pay for and never see.
- Retries. A failed tool call that triggers a retry doubles the cost of that step, silently.
None of this is waste in principle. It is how agents produce results that chat cannot. It becomes waste when nobody knows the per-run number and the architecture was never optimised for it.
Where the money actually goes
From the agentic implementations I have worked on, the distribution is fairly consistent.
| Cost driver | Typical share of spend | Usually reducible? |
|---|---|---|
| Re-sent context across agent iterations | 35 to 50% | Yes, substantially, via caching and context pruning |
| Over-powered model on routine steps | 20 to 35% | Yes, via routing |
| Retrieval payload larger than needed | 10 to 20% | Yes, via better chunking and reranking |
| Retries and failed runs | 5 to 15% | Partly, via error handling |
| Genuinely necessary frontier reasoning | 10 to 20% | No, and you should not try |
Read that table the right way. Roughly two thirds of a typical agentic bill is architecture, not intelligence. That is good news, because architecture is something you control without renegotiating anything.
The controls that work
Prompt caching first. If your agent resends a stable system prompt, tool definitions and retrieved documents on every iteration, caching that prefix is the single highest-return change available. It requires no model change, no quality tradeoff, and on a long-running agent loop it routinely removes a third of the bill. Most teams have not implemented it because it was not necessary at pilot volumes and nobody revisited it at production volumes.
Model routing second. The pattern that multiple enterprise deployments now confirm is routing roughly 80% of predictable requests to a small model and escalating the genuinely complex 20% to a frontier model. The cost differential is large enough to be worth engineering effort: processing 100 million tokens a day on a small model on rented hardware runs around $50 a day versus roughly $1,560 for a frontier model via API. I have set out when the small model actually wins and how to build the router in a separate piece.
Context pruning third. Agents do not need the full transcript at step forty. Summarising completed sub-tasks and dropping resolved tool output cuts the quadratic term directly. This is more engineering work than the first two and it carries a real quality risk if done carelessly, so measure before and after.
Cap the loop. Hard maximum iterations, hard maximum tool calls, hard maximum spend per run. Not because the cap will usually bind, but because the runaway agent that loops 300 times on an edge case is a real and recurring event, and without a cap you find out at invoice time.
Instrument per workflow, not per model. This is the one that changes conversations with finance. Cost per resolved support ticket. Cost per invoice processed. Cost per document reviewed. Those numbers can be compared to a fully loaded human cost and defended in a budget meeting. “We spent $41,000 on Claude last month” cannot.
What FinOps for AI looks like in practice
Ninety-eight percent of FinOps practitioners now manage AI spend, up from 31% two years earlier. That is one of the fastest scope expansions the discipline has had, and most teams are applying cloud-era habits to a problem that differs in three ways.
Attribution is harder. Cloud spend attaches to a resource with a tag. Token spend attaches to a request, and unless you propagate a workflow identifier and a team identifier through every call, the provider invoice tells you almost nothing. Instrument this on day one; retrofitting it across a live agent fleet is miserable.
Unit economics move. A cloud instance costs what it costs. An agent’s cost per run changes when you edit the prompt, when the underlying model updates, when a document gets longer. Your unit cost is a live number, not a rate card, and it needs monitoring rather than annual review.
The optimisation lever is upstream. In cloud FinOps, most savings come from procurement and rightsizing: reservations, commitments, instance types. In AI FinOps most savings come from engineering decisions made weeks earlier. Which means the FinOps function has to sit close to the build team, not next to procurement.
The number that should worry you
Only 39% of organisations can attribute any measurable business impact to their AI spend.
That is the actual crisis, not the token prices. A programme that costs 2.4x its budget but demonstrably removes 30% of a process cost is a good programme having a planning problem. A programme that costs exactly its budget and cannot show what it changed is a bad programme having no problem at all until someone looks.
I have written before about the honest numbers behind agentic AI ROI and about why 95% of AI pilots never reach production. The thread connecting both to this is measurement. Organisations that instrumented the workflow before they built the agent know their unit cost, know their unit benefit, and make cost decisions calmly. Organisations that built first are now reverse-engineering a business case from a provider invoice, which is the worst possible position to argue from.
If you are early enough to choose, instrument the process before you automate it. If you are not, start with caching and routing this quarter, and get a workflow identifier onto every request before the next budget cycle. Those three moves have paid for themselves in every implementation I have seen, and they are all engineering, which means you can start on Monday without asking anyone for permission.
Sources
- AI cost observability: measuring and justifying token spend in 2026, Vantage
- FinOps for AI overview, FinOps Foundation
- FinOps X 2026 recap: AI tokenomics explained, Mavvrik
- AI token costs: why enterprise AI bills keep rising in 2026, Optimum Partners
- Small language models eat the edge: the 32x cost disruption reshaping enterprise AI, Practical Logix