The July 2026 Model Wave: What Actually Changed After We Tested All of It

July 2026 was the densest model release month on record. Claude Opus 5, Sonnet 5 and Fable 5. GPT-5.6 in three tiers. Grok 4.5. Kimi K3 with open weights. Three Gemini models. Most of it shipped inside three weeks.

We spent nine days running eleven configurations through an identical battery of hard, script-graded tasks. No LLM judge, no rubric — each task has an answer a script checks.

Here's what we learned, including the parts that undercut our own methods.

1. On ordinary work, the frontier has converged

All eleven configurations passed all three hard tasks. Every one found an interval-merging bug that only surfaces when one interval is fully contained in another. Every one counted the letter "r" correctly in a phrase where the answer is 10. Every one produced four sentences of exactly five words without using a forbidden word.

That list spans a 17× price range, from Grok 4.5 at $0.0033 for the battery to Claude Fable 5 at $0.0561. It includes an open-weight model you can download, three budget tiers, and two flagships whose vendors advertise leads on a dozen public benchmarks.

We designed these tasks specifically to separate models, after an easier battery stopped discriminating. They didn't separate anything either.

The honest conclusion: for one-shot, verifiable production work — extraction, classification, debugging, structured output — price has stopped predicting capability. That is a real change from eighteen months ago, and it's the most consequential thing in the wave.

What this does not mean: our battery is single-turn and finishes in seconds. It cannot measure long-horizon agentic work, sustained tool-use chains, or multi-hour coding sessions, which is exactly where the expensive models are sold and exactly what their benchmark leads measure. Opus 5 leading on 9 of 12 published benchmarks is real. It just doesn't describe your extraction pipeline.

2. The rate card is a bad predictor of the bill

We measured this five separate ways, and it pointed the wrong direction every time:

The hidden variable is verbosity. Across the battery, output token counts ranged from 286 to 1,751 — a 6× spread for the same three answers. You pay for every token, so a terse model at a high rate routinely beats a wordy model at a low one.

Two of those rows are worth restating because they're counterintuitive in a way that costs money: Kimi K3 charges half of GPT-5.6 Sol's output rate and cost 54% more. Gemini 3.6 Flash charges more than Luna on both input and output and cost 32% less.

If you are budgeting from a pricing page, you are guessing. Run your own prompts and read the token counts.

3. Where you rent the model can matter more than which model

Kimi K3 is served by both Together and Fireworks at an identical $3/$15. Same open weights, same price.

Fireworks served it exactly 2.00× faster.

That's a larger effect than most model-to-model differences we measured, and it appears on no leaderboard, in no model card, and in no benchmark suite. If you've chosen an open-weight model and stopped there, you've made half the decision.

4. Rule-following runs inverse to price

The clearest separation we found all month came from the cheapest models.

Ten-run results on a trivially checkable constraint — three sentences, no letter "e":

The most expensive model in Anthropic's lineup honours this rule slightly less than half the time. Two of the cheapest models never broke it in thirty combined runs.

We are not claiming Gemini Flash is "better than" Opus 5 — they aren't sold for the same jobs. We are claiming something narrower and more useful: constraint adherence is an independent axis, it does not track price or benchmark rank, and if your product depends on a hard output rule you cannot infer reliability from tier. Validate the output. That works at every price point and costs nothing.

5. Most published comparisons — including some of ours — report noise

This is the finding we'd most like other people to take seriously, because it applies to us.

We measured Claude Opus 5 four times on the identical battery: $0.035025, $0.037125, $0.037225, $0.044200. A 26% spread, same model, same prompts. Any single-run comparison with a gap under roughly 25% cannot distinguish two models.

Worse, on the constraint test we ran Kimi K3 on two hosts. Same weights. One scored 5/5, the other 4/5. There is no mechanism by which a host changes a model's ability to avoid a letter — that difference was pure sampling noise, and it's the same magnitude as most "findings" in five-trial comparisons.

We published five-trial results early in this series and then stopped, doubling to ten trials once the evidence said five was insufficient. Our earlier numbers should be read as directional. So should anyone else's.

What we'd tell a team choosing today

Start cheap and prove you need more. The burden of proof has flipped. On verifiable one-shot work, assume the budget tier is sufficient until a measurement says otherwise.

Budget from measured tokens, never rate cards. Five for five, the price list misled us.

Pick the host deliberately for open-weight models. It was worth a 2× latency difference at identical cost.

Test constraint adherence separately. It doesn't correlate with anything else, and it's the axis most likely to break a product silently.

Trust vendor benchmarks for frontier-difficulty work. They measure something real that our battery cannot reach. Just don't let a benchmark lead decide a workload it never tested.

Run more than one trial. If your comparison hinges on a gap under 25%, you don't have a result yet.

Reproduce any of it

Every prompt in this series runs with no account and no API key:

• The interval-bug task
• The letter-counting task
• The constraint task — run it several times and watch it vary

The full cost table across all eleven configurations is in the August cost roundup.

To run any of this against your own prompts, side by side, with the token counts that drive the bill — that's Promptster.

This synthesis draws on runs conducted 2026-08-04 to 2026-08-09 via the Promptster API at max_tokens: 4000, reported across eight posts. Costs are computed from list pricing at time of writing against actual token usage. Latency figures are comparable within a post (paired, same session) but not across posts. Claude Sonnet 5 was measured at its introductory $2/$10 rate, which ends 2026-08-31. Every task is graded by script against a known answer; no model judged another. Where a result rests on a small sample, we've said so.