Inference Economics: Gartner's 96 Percent
Hardeep Singh, a senior research analyst at Gartner, put a number on what every cloud buyer already saw in their invoices1. Spending on AI-ready cloud infrastructure will climb 96% in 2026, to $42.276 billion, up from $21.529 billion1. Inference alone takes $23.3 billion of that, 55% of the total1. Gartner expects the share to reach 59% in 2027, as training's early lead fades1. Singh named the driver plainly. “This growth is driven by continued demand for infrastructure to support large language model (LLM) training and rapid operationalization of AI across enterprise applications”1.
Chart 1
Gartner forecasts AI-ready cloud spending climbs 96% in 2026, to $42.276 billion from $21.529 billion
Worldwide spending on AI-ready cloud infrastructure, Gartner forecast, in billions of dollars
- 2025
- 2026 forecast
- GPT-4, March 2023
- Open-weight models, 2026
- Gartner forecasts $42.276 billion in AI-ready cloud spending for 2026, up 96% from 20251.
- Inference takes 55% of that 2026 spending, rising to 59% in Gartner's 2027 forecast1.
- Data-center revenue at Nvidia hit $89.0 billion for the quarter ended late July 2026, up 117%2.
- Inference cost for the same capability fell a reported 95% over the two years into 20263.
- Fireworks AI reached a $17.5 billion value in July 2026, beside a reported $1 billion in yearly revenue4.
- Two rounds, $650 million and then $350 million, carried Groq to a $3.5 billion value in 20266.
- Cerebras shares rose 108% on their Nasdaq debut, then fell 10% weeks later on a margin warning78.
- BRAID's bounded-reasoning technique reported a peak performance-per-dollar gain of 74 times on one benchmark9.
- Adam Winter
- Andrew Feldman
- Armağan Amcalar
- Hardeep Singh
- Jensen Huang
- Lin Qiao
- Vipul Ved Prakash
- Agent Infrastructure
- Agent Infrastructure
- Agent Infrastructure
- Agent Infrastructure
Each column is one year's spending: the gray column is 2025, and the magenta column is Gartner's 2026 forecast, nearly twice as tall.
Source: [Gartner \[1\]](https://ailately.com/articles/inference-economics-gartner-96-percent#source-1). Chart by The AILately.com Desk.
The numbers behind this chart
| Item | Value | Note | | ------------- | -------- | ----------------------------------------------------- | | 2025 | $21.529B | | | 2026 forecast | $42.276B | Inference takes $23.3 billion of it, 55% of the total |
Nvidia's numbers match the scale. Data-center revenue hit $89.0 billion in the quarter ended late July 2026, up 117%2. Jensen Huang told investors what changed. "AI has reached its inflection point. It's doing useful work. Its tokens are productive and profitable. Now, compute is revenue."2 Read the last clause. A cost became a source of income.
A paradox that resolves through volume
Spending nearly doubling looks impossible beside a second trend running at the same time. The price of a token collapsed. One widely cited analysis puts the drop at roughly 95% over two years, and close to 1,000-fold over three3. GPT-4 charged $30 per million input tokens in March 20233. Open-weight models charged about $0.10 by 20263.
Chart 2
A million input tokens cost $30 on GPT-4 in March 2023 and about $0.10 on open-weight models by 2026
Price per million input tokens, in dollars
Each bar is one price for a million input tokens: the blue bar is GPT-4 in March 2023, and the magenta bar is the open-weight price by 2026, a sliver on the same scale.
Source: [Value Add VC \[3\]](https://ailately.com/articles/inference-economics-gartner-96-percent#source-3). Chart by The AILately.com Desk.
The numbers behind this chart
| Item | Value | Note | | ------------------------ | ------ | --------------------------------------------------------------------- | | GPT-4, March 2023 | $30.00 | | | Open-weight models, 2026 | $0.10 | One widely cited analysis puts the drop at roughly 95% over two years |
Both are true at once, and volume explains it. Companies now run far more inference calls than the price drop alone would predict. Agent work multiplies the calls one job sets off. A support agent handling a ticket in 2023 made one call. An agent system handling the same ticket in 2026 might make a dozen, each one sent to a sub-agent that checks a claim, reads a policy, or drafts a reply. Gartner's dollar figure measures that multiplication. The per-token chart measures only the price underneath it.
Fireworks and Together compete on serving speed
Lin Qiao built Fireworks AI on one bet. Serving open models faster and cheaper than the labs that trained them would win a market of its own4. The bet paid in scale this year4. A Series D announced July 15, 2026 valued it at $17.5 billion, beside a stated $1 billion in yearly recurring revenue4. Qiao led PyTorch at Meta before this, and her founding team came from the same place4. Dmytro Dzhulgakov and James Reed both worked on PyTorch core and its compiler4. Chenyu Zhao came from leading Google's Vertex AI platform4.
Together AI ran a parallel plan under Vipul Ved Prakash5. It raised an $800 million Series C in 2026, to press the case that open-source serving beats closed pricing at scale5. Ce Zhang serves as chief technology officer5. Tri Dao, known for the research behind faster attention, holds the chief scientist title5. That gives the company a team built to cut delay and cost out of open-weight serving5.
Groq's leadership change and Cerebras' margin warning
Money and a management change arrived together at Groq. The company raised $650 million in June 2026, then $350 million weeks later, reaching a $3.5 billion value6. Leadership changed hands between those raises6. Adam Winter, who joined in 2024 to run the international business, became chief executive in 20266. Groq's own leadership page confirms the move and skips the reason6. Winter now oversees chief financial officer Matt Eng, chief operating officer Alan Rice, and chief technology officer Sinclair Schuller6. Their custom chip claims to serve “millions of developers” running “trillions of tokens” weekly6.
Cerebras took the opposite path in public. Andrew Feldman, its chief executive and co-founder, rang Nasdaq's opening bell on May 14, 20267. A $5.5 billion raise sent shares up 108% on debut7. Five weeks later the stock fell 10%, when Cerebras forecast a thinner margin in its first earnings as a public company8. That warning matters past one company. It is public proof that the price collapse reaches the specialists too, even one running its own chips rather than reselling Nvidia parts8.
Bounded reasoning attacks the cost side directly
The inference specialists compete on serving efficiency. One research team went after something else: how much reasoning a model has to do per answer9. Armağan Amcalar, chief technology officer at OpenServ Labs, and researcher Eyup Cinar published BRAID9. It swaps open-ended chain-of-thought for structured, diagram-coded logic9. This edition's feature, “Reasoning's Rebate,” covers the paper directly9. Its headline result, 74 times the baseline on one test, works the demand side rather than the supply side Fireworks and Together fight over9. Amcalar summed up the aim himself. “BRAID boosts performance across every model class, from largest to smallest, making strong reasoning affordable”9. If the technique holds past benchmark conditions, Gartner's curve could flatten, because a real share of that $23.3 billion buys reasoning tokens a bounded approach skips19.
What the spending curve implies for agent infrastructure
Every agent framework, every payment rail for machine-to-machine trade, and every pilot reaching production needs inference cheap enough to run at scale. Model Context Protocol servers and Agent2Agent systems make calls at volumes a human-run workflow rarely reached. That volume is what Gartner measures, while the price per token keeps falling1.
Buyers reading the 96% figure alone will flinch. Read it beside the token-price collapse, and beside the churn among specialists racing to serve that demand cheaper, and it says something else. A market is still finding its scale, with price and volume pulling against each other, and the outcome open.
By the numbers
What to watch
Quarterly earnings from the specialists will show whether Cerebras' margin warning was one company's problem or the category's8. A second chipmaker reporting thinner margins would settle it. Gartner's 2027 inference share, forecast at 59%, deserves a check against real spending once the year closes, because agent volume could outrun even that1. Bounded reasoning reaching production outside benchmark conditions would be the clearest sign that cost, rather than price alone, is finally moving toward companies running agents at scale9.
Sources
1. "Gartner Forecasts Worldwide Artificial Intelligence-Optimized IaaS Spending to Grow 96% in 2026," Gartner, Aug. 10, 2026, <https://www.gartner.com/en/newsroom/press-releases/2026-08-10-gartner-forecasts-worldwide-artificial-intelligence-optimized-iaas-spending-to-grow-96-percent-in-2026>. 2. "NVIDIA Announces Financial Results for Second Quarter Fiscal 2027," Nvidia, Aug. 26, 2026, <https://nvidianews.nvidia.com/news/nvidia-announces-financial-results-for-second-quarter-fiscal-2027>. 3. "How AI Inference Costs Have Dropped 95% in Two Years, and What Happens Next," Value Add VC, 2026, <https://valueaddvc.com/blog/how-ai-inference-costs-have-dropped-95-in-two-years-and-what-happens-next>. 4. "Announcing our Series D and $1B ARR," Fireworks AI, July 15, 2026, <https://fireworks.ai/blog>. 5. "Announcing our $800M Series C to accelerate the shift to open-source AI," Together AI, 2026, <https://www.together.ai/blog>. 6. Groq leadership and news pages, Groq, 2026, <https://groq.com/about-us/>. 7. "Cerebras raises $5.5B, then stock pops 108%, in the first huge tech IPO of 2026," TechCrunch, May 14, 2026, <https://techcrunch.com/2026/05/14/cerebras-raises-5-5b-kicking-off-2026s-ipo-season-with-a-bang/>. 8. "Cerebras falls 10% after chipmaker forecasts shrinking margin in first earnings report since IPO," CNBC, June 23, 2026, <https://www.cnbc.com/2026/06/23/cerebras-cbrs-q1-earnings-report-2026.html>. 9. Armağan Amcalar and Eyup Cinar, "BRAID: Bounded Reasoning for Autonomous Inference and Decisions," arXiv, Dec. 17, 2025, <https://arxiv.org/html/2512.15959v1>.
Share
Copy AP citationCopy link
People in this piece
Companies
Gartner · Nvidia · Fireworks AI · Together AI · Groq · Cerebras · OpenServ Labs