Over Budget for AI in Finance? Here's What The CFO Needs to Know

Full transcript

Lightly edited for readability. The audio is the original.

0:0079% of companies blew their AI budget

79%. That's the share of companies that went over their AI budget last year. 500 finance leaders surveyed, and 79% of them blew the number. And the ones who rated themselves most mature at managing cost? Well, they did worse.

89% of them went over, by an average of 31%. Being good at this mostly meant they got a better view of it happening. Here's what I want you to sit with. That money went out the door one API call at a time, on a line item most finance teams have never forecast, never allocated, and in a lot of buildings don't technically own.

Today, I'm going to show you that line, why the number from your pilot was never going to hold, and the three things that move it, all of which you already know how to do. Because the formula underneath this one is a formula you've been running your entire career.

Welcome to Excel at AI in Finance, the show for finance pros who want to use AI to get home sooner and be the one their company can't afford to lose. I'm Mike, I run F9 Finance, and I'm a senior finance leader working in the trenches. All right, let's get into it.

1:19The $9,400 invoice nobody could explain

Let's make this real with something that happened. Names changed, and I've blurred a couple of details. This was back in February. A guy I'll call Taylor, a controller at a manufacturer, about 600 people, team of 9. He owns the close, the reporting pack, and the audit relationship. 11 years in the seat and very good at it.

And the setup was completely ordinary. That's the part I need you to hear. Nothing exotic was going on here. His FP&A group had started using an AI assistant to draft variance commentary. And that's the whole use case.

Somebody pulls the trial balance. The tool writes a first pass at the why for every account that moved more than 5%. A human edits it before it goes anywhere near the CFO. They'd been running it since October, saving about a day and a half a month, and everybody liked it.

The budget was $1,400 a month. That number came from the pilot, which ran 6 weeks in the fall and came in around $1,100. And then the number showed up. February invoice, $9,400.

Now, that wasn't a typo. That wasn't a duplicate charge. It wasn't someone's personal account getting billed to the company. $9,400 against a $1,400 budget, in a month where the team did exactly the same work they'd done in November.

So Taylor did what any of us would do. He went and asked. First stop, the analyst running it day to day. That analyst could say precisely what the tool did and how much time it saved him. He had no idea what it cost and no particular reason to.

Second stop, their IT manager. She could see the invoice, same as Taylor could. She could name the vendor it came from. She could not say what drove the increase, because the vendor's dashboard showed her tokens, and tokens meant nothing to anybody in the building.

Third stop, the vendor. And to be fair to them, they were helpful. They sent over a usage export: 40,000 rows, timestamps, and token counts. And here's the thing Taylor could not get. I want to be really precise about this, because it's the entire reason I'm doing this episode.

Nobody could tell him what one variance commentary cost. Not the vendor, not IT, not the analyst. He had a total and he had 40,000 rows. And he had no way to divide one by the other, because nothing in that file said, "This block of rows was one commentary for account 6200 in February."

Nobody in that building was doing anything wrong. Every person he talked to was competent, and every one of them was telling him the truth as they understood it. There was just no meter on the thing. The work was getting done. The money was going out. And those two facts had never been introduced to each other.

It took Taylor 6 weeks to get an answer, and the answer was completely mundane. In January, somebody had improved the prompt. They started attaching the prior 3 months of commentary so the tool could match the house style. And it worked. The output got noticeably better.

It also meant every single request was now dragging 3 months of documents behind it, on a tool that resends everything every time. Better output. Same number of runs, close to 7 times the bill. And the person that made that change had no way of knowing, because nothing in their workflow ever showed a price.

The fix took an afternoon once they understood it. They cached the style examples instead of reattaching them, and March came in at $2,200. But the part that stuck with me is the shape of it. The fix was easy. Finding out took 6 weeks. And inside those 6 weeks, Taylor's CFO had already asked him twice to explain a variance he had no ability to explain.

5:47How AI spend became a line on the finance budget

Here's what changed. Two years ago, AI spend was an experiment on somebody's corporate card. Somebody in engineering had an API key, the bill was four figures, and finance had no particular reason to look at it. Now it's a line all on its own.

The FinOps Foundation runs the biggest survey in this space. Roughly 1,200 practitioners, covering $83 billion of cloud spend between them. In 2024, 31% of them said they manage AI spend, or expect to within a year. Last year? That was 63%. This year, it's 98%. That is the fastest expansion of scope they have ever recorded.

And Deloitte went and asked 200 CFOs at billion-dollar-plus companies what worries them most about AI inside their own business. The top answer, at 46%, was cost uncertainty and cost transparency. Second was confidence in their AI operations, at 35%. Third was AI-driven mistakes, at 25%. Cost beat both of them.

So the line arrived, and the CFOs already know they can't see it. Three things are true about this line that aren't true about any other software line on your budget, and each one is a reason your forecast is wrong. I'll walk all three, and then I'll give you three questions that start fixing it.

7:06Why nobody owns the AI cost line

The first problem is ownership, and this number arrived without one. DoiT ran a survey with Sapio Research: 500 finance leaders, US and UK, companies over 1,000 people. Now, quick flag on this: DoiT sells cost tooling, so take the framing with the appropriate grain of salt. But the fieldwork was independent and the questions were good.

They asked who is accountable for AI spend. Technology said 55%. Finance said 53%. Read those two together. Both functions think they've got it, which means on any given Tuesday, neither one is doing it. Their phrase for that was better than anything I'd have come up with: split means unowned.

And here's why it stays split. The people who have the tooling to see this don't work for you. In the FinOps Foundation data, 78% of FinOps practices report to the CTO or the CIO. 8% report to the CFO, and that 8% has been shrinking. So the visibility sits in one org, and the accountability sits in yours. That's the problem in one sentence.

Flexera put a number on the visibility gap in July. They asked 512 IT asset managers how much visibility they have into AI software spend. 31% said yes, they have accurate visibility. On-prem software, for comparison, is 78%. SaaS is 66%. AI is the worst-lit corner of the entire technology estate.

And 59% of them said wasted AI software spend went up over the last year. Worst of any category they track. Public cloud software was 44%. Desktop software was 23%. Now watch what that does to a budget.

a16z surveys about 100 CIOs at Global 2000 companies every year. There's a chart in this year's deck titled, and I promise I'm not making this up, "Actual spend is higher than already bullish forecasts." Enterprises budgeted around $3.9 million for AI applications. They're spending $5.8 million. Roughly a 50% overshoot, on a budget they set themselves, in a category they chose to enter.

And then back to that maturity paradox from the top of the show, because it's the beat that should bother you most. In the DoiT data, the companies that rated themselves very mature or leading edge on cost management had an 89% overrun rate, an average 31% over. The early stage companies came in at 69%, 16% over.

DoiT's read on that was sharp. Maturity is a precondition for knowing what's happening. It's not a protection against it happening. Which is a polite way of saying those mature companies might be overspending by about the same amount as everybody else. They're just the only ones who can actually see it.

The fix on this one is boring, and it's organizational. Somebody, one somebody, has to own this number by name. And the only function that has ever been good at owning a number that nobody else wants is finance.

10:10Why AI cost grows like a triangle, not a line

Part two. This one's mechanical, and it's the part that changes how you forecast. You're billed per token. A token is roughly three quarters of a word. Two things about that meter surprise every finance person I've ever explained it to.

First one: output costs more than input. On Claude, output tokens are exactly 5 times the price of input tokens. Every model on the page, no exceptions. On the main GPT-5 models, it's roughly 6 times. So the same model giving you a long answer costs multiples of that model giving you a short one. Verbosity is a rate decision, and it almost never gets treated like one.

Second one, and this is the big one: the model has no memory on its own. Every time you send a message in a conversation, the entire conversation goes with it. All of it. Every earlier question, every earlier answer, every file you attached at the start. OpenAI's own documentation says it flat out: "All previous input tokens in the chain are billed as input tokens."

So think about what that does to the math. Turn 1, you send 1 unit of context. Turn 2, 2 units. Turn 3, 3. By turn 10, you're sending 10 units on that one message alone. Add it up across the whole conversation, and a 10-turn job comes to 55 units of context, at the point where your intuition says 10.

The cost grows like a triangle instead of a line. If you take one thing out of this episode, take that. A triangle, not a line. It is the reason the number from your pilot was never going to survive contact with production.

Because in a pilot, somebody runs 3 or 4 turns by hand and reports what it costs. In production, an agent runs 30 turns to finish a job, unsupervised, and every single one of those turns is dragging the whole file behind it.

Anthropic published their own multiples on this, which I appreciate more than I expected to. A single agent uses about 4 times the tokens of a chat conversation. A multi-agent setup uses about 15 times. And, their words, not mine: "For economic viability, multi-agent systems require tasks where the value of the task is high enough to pay for it." That's a vendor telling you to build a business case. Take the hint.

And the volume moves underneath you even when nobody changes anything. Datadog has telemetry across more than 1,000 customers running this in production. Average tokens per request more than doubled year over year for the median customer. For the heaviest 10%, it quadrupled. Nobody announced a price increase. The work people are feeding AI just got heavier.

Then there's the part that never appears on the invoice at all. The FinOps Foundation's guidance puts it at 40 to 60%. 40 to 60% of what a feature costs in total sits in the plumbing around the model. That's vector databases, embeddings, re-ranking, orchestration, caching layers, data transfer between regions, observing and auditing.

None of that shows up on the bill from Anthropic or OpenAI. So when somebody hands you a cost per query built off the API invoice, it's understated by roughly half. That's a scope error, and it's in every AI business case I've been shown this year.

13:27Uber and GitHub: when the unit economics break

Let me give you two examples of this landing on real companies. Uber burned through its entire 2026 AI budget in 4 months. 4 months into a 12-month year. That came from their own CTO, and the reporting says they'd helped it along by putting up internal leaderboards, ranking teams on how much AI they used.

Then their president, Andrew Macdonald, said the hard part out loud on a podcast in May. His words: it's very hard to draw a line between one of those stats and, quote, "Okay, now we're actually producing like 25% more useful consumer features."

Second one: GitHub, back in April, announced a repricing. Their chief product officer wrote this on the company blog, and it's the most candid thing a public company has said about any of this. The quote is, "Today, a quick chat question and a multi-hour autonomous coding session can cost the user the same amount. GitHub has absorbed much of the escalating inference cost beyond that usage, but the current premium request model is no longer sustainable."

That is a vendor telling you their unit economics broke. And what they shipped to fix it was billing on input, output, and cached tokens at published rates, which means their volume risk just became your volume risk. So price the work at production volume and production turn count, because the business case you approved and the business case you're going to run are two different documents.

14:50Lever 1: prompt caching at 90% off

Three levers, and every one of them is a boring finance lever. Lever one is caching. If a chunk of your prompt repeats between calls, and in finance work it almost always does (the same system instructions, the same chart of accounts, the same policy document), the provider will store that chunk and charge you a tenth of the normal rate to read it back.

Anthropic publishes the multiplier right on the pricing page. Cache reads are 0.1 times base input. 90% off on the single biggest volume driver in the whole bill. Now here's how much of that is sitting on the floor.

In Datadog's telemetry, across the models that support caching at all, only 28% of calls show any cached tokens. The other 72% are paying full freight on text they already sent.

Ramp sees the actual invoices for over 1,000 businesses. They found companies running the same model paying an effective $0.62 per million tokens, against a $3 list price. Same model, same vendor, 5 times apart, and the entire gap is caching.

And the reason I'm giving you this lever first is that a finance person found it. Ramp put a quote on their own product page from Greg Cooley, the controller of AngelList. Now, this is marketing, so weigh it accordingly. He said prompt caching wasn't something on his radar as a controller. He routed it to engineering, and they found they'd been losing $10,000 a month. They fixed it the same day. He just needed to know to ask.

16:20Lever 2: model routing

Now, lever two is routing. Most of the work does not need the expensive model. Cursor ran this as a real A/B test across millions of requests and published 60% savings at the same quality. And separately, they published cost per commit: $6.70 routed, against $12.60 on their most expensive model.

But their diagnosis is where the finance lesson is. Roughly 60% of their developers picked one model as a daily driver and never changed it. So routine work was running at frontier prices all day long, because of a default that nobody went back and questioned.

Ramp says routing cut its own AI costs by 30%. Now, that's a procurement problem wearing a technical costume, and you've solved the shape of that problem before.

17:08Lever 3: the 50% batch discount

Lever three is the batch discount, and it's the easiest thing you will ever get approved. Anything that doesn't need an answer in the next hour runs at 50% off the token price. Anthropic, OpenAI and Google, same discount, published on their pricing pages. Check which models are eligible, because not all of them are.

Overnight reconciliations, bulk transaction categorization, document extraction, anything that can run while everybody's asleep. Half price for patience. That's the lever.

17:393 questions to ask about AI cost this week

So what do you do this week? Three questions, and you can ask all three standing in a hallway.

Question one: what does one unit of this work cost us? Not the monthly total. One close, one reconciliation, one resolved ticket, one forecast run. If nobody can answer that, stop right there, because you've already found the problem.

Question two: what percentage of our input tokens are cache reads? You don't have to understand how caching works to ask it. If the answer is a blank look, or anything under 50%, there's money on the floor and somebody can pick it up this week.

Question three: which model is each step running on, and who chose it? Ask that one to find out whether a decision got made, or whether a default just won by never being questioned.

Three questions, and none of them require you to be technical. And all three of them are questions that only finance is going to think to ask.

Every AI bill comes down to three things: how many turns the work takes, which model each turn runs on, and who's accountable for both. That's volume, that's rate, that's ownership. You've been forecasting volume and rate your entire career. And ownership is the thing you're already the best in the building at holding on to.

This line showed up wearing a technology costume. Underneath, it's a finance problem you already know how to run.

19:04Build the meter before you build the model

So here's the line I'd hold. Build the meter before you build the model. If you can't tell me what one unit of this work costs today, a 12-month forecast is a guess with a spreadsheet wrapped around it.

Get the meter in. Watch it for 60 days, then set the budget at 110 to 120% of what you measured. That's the FinOps Foundation's own recommendation, and it's the least glamorous and most correct answer in the entire episode.