| Key Takeaways
• The price of reaching a given level of AI performance has fallen sharply, yet enterprise spending on model APIs more than doubled in six months.[1][2] • Most organizations cannot say who owns the AI bill or why it spiked.[3] • Organizations can regain control by routing every LLM API call through a single gateway that assigns ownership, enforces budgets, and records usage by department and model. |
AI tokens have become dramatically cheaper. Epoch AI found that the price of reaching a given level of performance fell between 9 and 900 times per year, depending on the task. The price of matching GPT-4’s performance on PhD-level science questions, for instance, fell 40-fold per year.[1]
The bill that companies pay, however, tells a very different story. Menlo Ventures estimated that enterprise spending on model APIs more than doubled in six months between 2024 and 2025.[2] And for many organizations, that bill now arrives as a surprise. In Harness’s 2026 survey, 72 percent of engineers had experienced an unexpected AI cost spike or bill in the past year.[3] A separate survey of 500 finance leaders, commissioned by DoiT, found that 79 percent had experienced AI cost overruns over the same period.[4]
This raises the question: if the price of AI keeps falling, who is accountable for the bill that keeps rising?
Who Can Answer for These Bills?
More than half of Harness respondents said no department seems clearly accountable for AI costs. Responsibility is split across engineering, FinOps, finance, and IT.[3] DoiT found the same pattern, with accountability for AI spending being divided almost evenly between technology leadership and finance. The lack of a clear single owner at the operational level means organizations lack insight into where and how widely its API keys have been distributed. Nor do they know which teams are using which models and how much.[4]
Policy has the same limitation. Harness found that 73 percent of organizations have AI cost policies in place, yet only 13 percent have basic visibility into their AI spend.[3] The problem is that an organization cannot enforce a policy on AI cost overruns if it does not even have basic visibility into who is spending how much.
Seeing an Overrun Is Not the Same as Preventing It
But visibility alone is not enough, either. In DoiT’s survey, 89 percent of organizations that rate themselves very mature on FinOps experienced AI cost overruns. DoiT’s interpretation is that mature programs are better at surfacing overruns, not at preventing them.[4] A report that sees spending only after the invoice arrives can describe an overrun but not stop one. Prevention requires a control that acts on the API call itself, before the cost is incurred.
Stop Handing Out Keys. Build a Gateway Instead.
When API keys are scattered across teams and systems, cost management breaks down. And reporting only after a cost overrun has already occurred is not the same as prevention.
The solution is to route every LLM API call through a single gateway your organization owns. With every request and response flowing through one control point, you can inspect exactly what data leaves your organization and control what department is spending how much. You can also block usage once spending surpasses a threshold.
AI-R Gateway (AI Radar Gateway) brings all your organization’s LLM API usage under one roof.
- Virtual keys allocated by department. Issue virtual keys by department or project without exposing your actual API keys. Reduce the risk of data leaks while ensuring every call traces back to a clear owner.
- Usage and cost visibility. Track API call volume and cost by department and model in real time and analyze which systems are calling which AI models. You get a transparent picture of where your AI budget goes.
- LLM API budget control. Set a budget limit for each virtual key and get alerted as usage nears the threshold. Once the limit is exceeded, usage is blocked automatically. No more surprise invoices at the end of the month.
- Sensitive data leakage prevention. Scan every outbound request and every inbound response for sensitive information. Block, warn, or take other action based on your organization’s security policies and compliance standards. Because API traffic moves from system to system without passing through a user’s screen, the gateway is the point where that data can be verified.
- No change for the teams using AI. Departments and systems keep calling APIs exactly as they do today, with no changes on their end. Every usage record is centralized in one gateway, giving you full AI governance without compromising productivity.
Conclusion
Falling prices were supposed to make AI spending easier to manage. Instead, the cost of a token has dropped while total spending has climbed. Most organizations cannot say who owns the bill or why it spiked. Organizations that route every API call through a gateway can gain visibility into their API costs and prevent overruns. They can assign each call to an owner, see spending as it happens, and stop it at a limit they set.
See how AI-R Gateway brings LLM API costs and security under one control point. Book a demo.
Sources & References
[1] Ben Cottier, Ben Snodin, David Owen, and Tom Adamczewski, “LLM inference prices have fallen rapidly but unequally across tasks,” Epoch AI Data Insight, March 12, 2025: epoch.ai
[2] Menlo Ventures, 2025 Mid-Year LLM Market Update: Foundation Model Landscape + Economics, July 31, 2025: menlovc.com
[3] Harness, “New Harness Report Reveals Enterprise AI Spend Has Outgrown the Systems Built to Track It,” press release on the 2026 State of AI in FinOps report, July 29, 2026: harness.io
[4] DoiT, “The AI Spending Data Your Board Is About to Ask You About,” June 9, 2026, reporting a survey conducted by Sapio Research in February 2026: doit.com
This article is general information about AI cost management practice, not financial advice. The DoiT, Harness, and CloudZero surveys cited here were commissioned or published by vendors of cost management tools; readers should consult the full reports for methodology. Epoch AI notes that the fastest price declines in its data occurred within a single recent year and may not persist. Survey figures reflect the study periods stated in the underlying research and may not describe current conditions.