Agent business cases share a family resemblance. Volume times handling time times labour cost, multiplied by an automation rate, minus inference spend. The figure is large, the payback short, the project approved.
Twelve months later the savings are hard to locate. The system works, users are broadly positive, and nobody can point to the line where money appeared. Usually the model was not wrong about automation. It was wrong about everything surrounding it.
The costs that get left out
Human review
Nearly every enterprise agent has a human in the loop for some fraction of cases. If 90% of outputs are auto-approved and 10% reviewed, the reviewer’s time is a running cost of the system. Where review demands more senior judgement than the original task, the per-case cost of the reviewed fraction can exceed the cost of the original process.
In regulated contexts this is not optional overhead. The RBI’s FREE-AI framework expects human oversight and grievance redressal for AI-influenced decisions; that staffing is part of the system’s cost, not an adjacent compliance line.
Exception handling
What happens to cases the agent cannot complete? If they queue for specialists, the queue is a cost. If the agent partially completes and a human finishes, the handover frequently costs more than doing the task from scratch — reconstructing what the agent did and why is not free, and is harder when trajectories are not well instrumented.
Maintenance
Agent systems need continuous attention: models change, prompts drift out of alignment with evolving processes, tool interfaces update, evaluation sets need extending. Budget engineering capacity as a permanent line, not a project cost. Systems treated as finished degrade quietly — the failure mode is silent, which is precisely why it goes unbudgeted.
Trajectory cost, not token cost
Most models price inference as tokens per call. Agents make many calls per task, plus tool invocations and external API charges. A well-designed pipeline tracks cost at the trajectory level, which lets you identify which steps are disproportionately expensive.
This is where framework choice quietly shows up in the P&L. Benchmarking of identical multi-agent workflows has found token-efficiency spreads of 8–9× between orchestration frameworks, driven by how each handles context aggregation. That is a cost-structure decision disguised as an architecture decision.
The cost of being wrong
Most models assume errors are free. A wrong answer to a customer may cost a complaint, a remediation, or regulatory exposure. Estimate the error rate, estimate cost per error, and put the product in the model. For consequential decisions this term sometimes dominates everything else — and under the DPDP Act, with penalties reaching ₹250 crore per violation, the tail is not theoretical.
The baseline problem
Here is the failure that undermines more agent ROI claims than any cost omission: nobody measured the process properly before automating it.
Absent a baseline, post-deployment improvement is unfalsifiable. Handling time may have fallen because the agent helped, or because volume mix shifted, or because a different team absorbed the hard cases, or because someone changed the measurement definition. Everyone attributes the gain to the project. Nobody can prove it.
Measure first: volume, handling time distribution rather than the mean, error rate, escalation rate and cost per case, for at least a month before anything changes. The distribution matters because agents typically absorb the easy cases, which lifts the average human handling time afterwards and makes the comparison look worse than reality.
The metric worth managing
Fully loaded cost per successfully resolved case, compared against the same figure for the pre-agent baseline.
Fully loaded means inference, tool and API charges, review, exception handling, and amortised maintenance. Successfully resolved means the case actually ended satisfactorily — verified on end state — not that the agent produced output.
This number resists most of the ways agent ROI gets overstated, because it cannot be improved by shifting work elsewhere or by producing more output of lower quality. An agent at 92% accuracy and $0.03 per task is a fundamentally different proposition from one at 94% and $1.50, and only the combined metric makes that visible. Plot accuracy against cost for every version.
Value that is real but sits outside the metric
Cost per resolved case is a good primary metric and an incomplete picture. Genuine benefits it does not capture:
- Latency improvements that change customer behaviour
- Capacity to absorb volume spikes without hiring
- Consistency in regulated processes, which reduces variance rather than cost
- Work not previously done at all because it was uneconomic
Track these separately and describe them as strategic rather than folding speculative figures into the financial case. A business case claiming a defensible number and naming its uncertainties survives scrutiny far better than one claiming a large number that cannot show its working.
A short checklist
- Baseline the existing process for at least a month — distribution, not just averages.
- Model review, exception, maintenance and error costs explicitly.
- Instrument cost at trajectory level, not per call.
- Track fully loaded cost per successfully resolved case as the headline number.
- Verify “resolved” against end state rather than agent self-report.
- Report strategic benefits separately and qualitatively.
- Re-measure quarterly — agent economics move as models, prices and processes change.
Frequently asked questions
How do you measure AI agent ROI?
Fully loaded cost per successfully resolved case, compared against a pre-deployment baseline for the same process. Fully loaded includes inference, tool charges, human review, exception handling and amortised maintenance. Resolution should be verified on end state rather than agent self-report.
Why do AI agent savings fail to materialise?
Usually because the business case omitted human review, exception handling, maintenance and error costs — and because no baseline was captured, making improvement impossible to attribute or prove.
What should we measure before deploying an agent?
Volume, handling time distribution, error rate, escalation rate and cost per case, over at least a month. Distribution matters because agents typically absorb easier cases first, which raises average human handling time afterwards and distorts naive comparisons.
