I Tested ChatGPT 5 Versus Claude Opus on Real Finance Work

Every week I get some version of the same message from finance pros: which AI model should I actually be using? Not which one writes the best blog posts or scores highest on math olympiad problems — which one holds up when there’s a variance report due Friday and a CFO waiting for commentary.

I ran both new flagship models through three tests this week. Same dataset, same prompts, side by side. Here is what I found.

The Two Models Everyone in Finance Is Asking About

Claude Opus 4.8 launched May 28th. ChatGPT 5.5 dropped April 23rd. Both cost around $5 per million input tokens. Both support a 1 million token context window. Both are being marketed as the best AI model available right now.

The general tech coverage focused on coding benchmarks, mathematical reasoning, and long-context retrieval. Useful if you are a developer. Less useful if you are an FP&A manager trying to figure out which tab to open before your next close.

Finance pros need a different comparison. The relevant questions are: which model produces more accurate output when given financial data? Which one handles multi-step analysis? Which one writes commentary that sounds like a finance manager wrote it and not a press release? I had a client — director of finance at a consumer goods company — ask me this exact question last month. She had been using both tools on rotation and couldn’t tell which was actually better. That is what this article is about.

ChatGPT 5 Versus Claude Opus Benchmarks For Finance

Most model comparisons lead with MMLU or HumanEval scores. Here is the one that matters for finance work.

Benchmark	Claude Opus 4.8	ChatGPT 5.5
Finance Agent v2	64.4%	60.0%
Hallucination rate (finance/legal/medical)	36%	86%
Terminal-Bench 2.0	69.4%	82.7%
BrowseComp	—	90.1%
GPQA Diamond	94.2%	—

Finance Agent v2 tests autonomous multi-step financial analysis: planning, tool use, and coherent output across tasks like building financial models, running variance analysis, and producing board-ready presentations. It is the closest thing to a benchmark for what finance pros actually do.

The hallucination rate number deserves its own sentence. At 86%, ChatGPT 5.5 produces confident incorrect answers on high-stakes financial, legal, and medical prompts more than 8 times out of 10 in benchmark conditions. Claude Opus 4.8 comes in at 36%. That is not a rounding error. That gap changes which model you trust with a report that goes to a finance leader.

The key insight: A benchmark tells you what a model does across thousands of prompts under controlled conditions. It does not tell you what happens on your file, with your account structure, on a Tuesday afternoon when the number is due in ten minutes. That gap between benchmark and reality is exactly what I went looking for — and what I found was not what I expected.

Test 1: Cross-File FP&A Reasoning

I used two files from my Coffee Shop demo dataset: a monthly P&L (six months of actuals and budget across three locations — Astoria, Hell’s Kitchen, Lower Manhattan) and a point-of-sale transaction file with approximately 149,000 rows.

The setup

To keep this fair, I ran both in their web apps — no desktop advantages on either side. I set Claude Opus 4.8 to its highest reasoning tier (high effort) and gave ChatGPT 5.5 thinking mode, which is the closest equivalent setting on its end. Same data, same prompt, same level of effort requested from both.

What I asked both models

I've uploaded two files: a monthly P&L for January through June 2023 and a
transaction-level POS file with approximately 149,000 rows. Both cover three
coffee shop locations: Astoria, Hell's Kitchen, and Lower Manhattan.

I have one marketing budget allocation decision to make for Q3. I can only
increase spend at one location.

Using both files together, tell me:
1. Which location has the strongest revenue per transaction trend and what's
   driving it (look at product mix and peak hour concentration)?
2. Where is the biggest gap between budget potential and actual performance —
   and does the operational data support closing that gap?
3. Give me your recommendation: one location, and the specific insight from
   the data that makes you confident.

Do the actual cross-file analysis. Don't summarize each file separately and then guess.

What came back

Here is what makes this test useful: I built this dataset, so I know the right answer going in. Astoria is the location that has not reached its potential — light morning peak load, a strong operating margin, and marketing spend that was already underweighted relative to the other two. Hell’s Kitchen already received its investment and is performing close to its ceiling. Lower Manhattan is saturated; there is nowhere left for new marketing dollars to go.

Claude recommended Astoria. It got there by reasoning forward — connecting Astoria’s lighter traffic and healthy margins to genuine headroom, and explicitly calling out that its marketing budget was underspent relative to the other locations. Just as useful: it narrated its reasoning as it worked, laying out each step of the cross-file analysis — tying POS transaction patterns to the P&L variance — in a way I could follow and check. It took about five minutes.

ChatGPT recommended Hell’s Kitchen. The analysis itself was not bad. It correctly identified Hell’s Kitchen’s scale, transaction growth, and budget conversion, and it walked through the numbers competently. But it leaned almost entirely on what had already happened rather than what could happen next, which is exactly backwards for a forward-looking budget call.

The location it picked had already gotten its investment and proven its ceiling. The location that actually had room to grow never made the shortlist. It took about six minutes, and unlike Claude, it gave me very little visibility into how it got there, mostly status updates that it was “running the analysis.”

This is the test that matters most for FP&A work: not whether a model can read a P&L, but whether it reasons about where a business is headed instead of just where it has been. On a decision I had already worked out the right answer to, only one model got there.

Test 2: Numerical Accuracy Under Pressure

This is the test that matters most if you are sending AI output to a finance leader — and it produced the most useful surprise of the entire comparison.

Why hallucination rate matters in finance

Hallucination in a general context means the model makes something up. In a financial context, it means the model writes a specific dollar amount or percentage that does not match the source data — with the same tone and confidence it uses when it is correct. A controller I worked with last year found an invented variance figure in AI-generated commentary two days before the board deck was due. The number was plausible. It passed a quick read. It was $40,000 off.

The exact prompt

I'm attaching a monthly P&L file. I need exact figures — show your calculations.

1. What was the dollar variance between Actuals and Budget for Sales Revenue
   at Astoria in May 2023? Calculate it as Actuals minus Budget and give me
   the percentage too.
2. What was Hell's Kitchen's Labor cost as a percentage of its Sales Revenue
   for April 2023? Use the Actuals figures only.
3. What was the total Actual Sales Revenue for Lower Manhattan across all six
   months (January through June 2023)?
4. Which single month had the largest favorable Sales Revenue variance across
   all three locations combined? Give me the month, the location, and the
   dollar amount.

Show every calculation step. Do not estimate or round — give me the exact figures.

What happened

Both models got every figure exactly right.

Four questions, each requiring a different kind of calculation: a dollar variance with a percentage, a labor-cost ratio, a six-month revenue total, and a multi-step scan across every month and every location to find the single largest favorable variance anywhere in the dataset. I checked all four against the source file by hand. Claude and ChatGPT matched it, down to the decimal, on every one.

That is not the result I expected walking in, given the 86% vs. 36% hallucination gap above — and it turned out to be the more useful finding, not a less useful one. Here is why: that benchmark gap shows up under high-stakes, ambiguous, multi-step conditions. My Coffee Shop dataset has one header row, consistent account names across every period, and no merged cells or stray subtotals. It is about as clean as financial data gets, and clean data is exactly where the gap between these models narrows to nothing.

Most files that land on a finance team’s desk are not that clean. Account names drift between periods. Subtotals get mixed in with detail rows. Date formats change halfway down a column. That is where the published hallucination numbers almost certainly start to separate the two models — and it is precisely the kind of file this test did not use.

The honest takeaway: on a clean, well-structured file, I would not lean on the hallucination benchmark to pick a model. Both handled it identically here. The benchmark becomes relevant the moment your data gets messy, which — if you work in finance — is most of the time.

How to run this test on your own data

You do not need the Coffee Shop dataset, and you should not use your cleanest file. Run this on the file you complain about, the one with inconsistent account names or a merged header row. That is where you are far more likely to actually see the gap the benchmark describes, and it is the file that matters most anyway. Adapt this prompt template to whatever P&L or variance file you work with regularly:

I'm attaching a financial data file. I need exact figures — show your calculations.

1. What was the dollar variance between Actuals and Budget for [Line Item]
   at [Location/Department] in [Month]? Calculate as Actuals minus Budget
   and give the percentage.
2. What was [Location/Department]'s [Line Item] as a percentage of
   [Revenue Line] for [Month]? Use Actuals only.
3. What was the total Actual [Line Item] for [Location/Department] across
   all [X] months?
4. Which single [period] had the largest favorable [Line Item] variance
   across all [locations/departments]? Give the period, the entity,
   and the dollar amount.

Show every calculation step.

Three rules for adapting it effectively:

1. Ask for an exact figure, not a range or approximation
2. Ask the model to show the calculation, not just the answer — this forces it to expose its reasoning
3. Include at least one question that requires multi-step math (a percentage that needs both summing and dividing)

Test 3: From Raw Data to Board-Ready Output

The final test is the one that shows up most often in real close cycles.

The request

I loaded two files into both models: the P&L GL data and a commentary file containing real management notes written by the finance team for May 2023 across all three locations — operationally specific, the kind a finance manager actually writes. I asked both models to read the May commentary for style and tone, then write June commentary for all three locations grounded in the actual June figures.

What came back

Worth saying plainly, because it is exactly the kind of thing you would only catch by checking: both models took some liberties with the source commentary rather than staying tightly anchored to it. That alone is a reminder to read AI-drafted commentary against the notes it was supposed to be matching, not just against the numbers.

That said, Claude’s version read closer to what the finance team would have actually written. It cited specific figures, tied labor efficiency to schedule adjustments, and — without being asked — flagged that Lower Manhattan’s rising cost of goods sold was something worth investigating. That last part is the kind of forward-pointing detail that turns commentary from a recap into something a finance leader can act on.

ChatGPT’s version was accurate at the surface level but stayed more generic. It restated what the numbers said without picking up much of the operational context that the source commentary actually carried. Both drafts needed a human review pass before they were ready to send. Claude’s needed less editing to get there.

ChatGPT also finished noticeably faster on this round, about a minute and seventeen seconds, versus roughly two minutes for Claude. If speed on a first draft is what you care about most, that is a real point in ChatGPT’s favor.

The instruction-following test

After the full commentary output, I sent this to both:

The CFO needs the top 3 business issues from June in bullet form. 3 bullets only.
25 words maximum per bullet. Start each bullet with the location name.
No intro sentence, no conclusion.

Both models followed the format exactly: three bullets, location name first, no intro, no conclusion, all under the word limit. Where they split was on substance.

ChatGPT’s bullets restated the obvious — “cost of goods sold was unfavorable despite strong revenue growth” for Astoria. True, but it tells a finance leader nothing they could not already see on the P&L itself.

Claude’s bullets named the actual driver behind each number: rising dairy and pastry costs at Hell’s Kitchen, input costs and waste eating into margin at Astoria, the largest month-over-month cost-of-goods-sold jump of the three locations at Lower Manhattan. That is the difference between a summary and an insight — and it is the difference a CFO actually notices when commentary lands on their desk.

For a process that runs twelve times a year, that gap compounds. A model that hands you insight instead of restatement is the one that saves you the most time on the edit pass.

My Recommendation

Here is the rule I am using starting this week: open Claude first for core finance output — variance analysis, cross-file reasoning, commentary, anything that is heading toward a finance leader’s desk. Open ChatGPT for research and build-type work, where speed and breadth matter more than precision on a single number.

That is not a knock on ChatGPT 5.5. It is a strong model, and it matched Claude stride for stride on raw numerical accuracy in this test, and it was faster on a couple of the rounds. But on the two tests that matter most for finance work — reasoning toward a forward-looking decision, and producing commentary a finance leader can use without a rewrite — Claude was the more reliable starting point. It got the harder call right in Test 1, when ChatGPT got it wrong. It needed less editing in Test 3. And it showed its reasoning as it worked, which matters when someone asks you why you trust the number.

Neither model replaces your judgment, and they are not fully interchangeable — they cover different parts of the workflow well. But if I had to pick the one tab to open first when a number is about to land on someone else’s desk, it is Claude.

How to Run This Test on Your Own Data

Before you commit to either model for close work or variance reporting, run the accuracy test yourself. It takes 10 minutes on a file you already have.

Step-by-step:

1. Open Claude Opus 4.8 and ChatGPT 5.5 in separate tabs
2. Upload the same P&L or variance file to both
3. Send the four-question accuracy prompt above (adapted to your own line items and locations)
4. When responses come back, open your source file and check each figure
5. Note any discrepancy: model’s number alongside the file’s number
6. Send a follow-up with a tight constraint: “Rewrite this as three bullets, 25 words max each, location name first, no intro”
7. Check whether both models follow the constraint exactly

If both models come back clean, that is not a green light to stop checking — it likely means your file is cleaner than most of what comes across your desk. Run it again on a messier file before you draw any conclusions. And if you do find an error: adjust the workflow, not just your trust in the tool. AI earns its place in drafting, structuring, and first-pass analysis. Human review stays on any specific figure going to a finance leader. The test tells you which model — and which kind of file — needs the closer look.