Open-Weight vs Closed-Source, August 2026: Parity on Capability, and Open Isn't Always Cheaper

We first ran this comparison in April. The landscape has changed enough to redo it: Kimi K3 shipped with open weights and a 1M context window, GLM 5.2 and GPT-OSS 120B are widely served, and the closed side turned over almost entirely in July.

Three open-weight models, one closed flagship, the same four hard tasks we've used across this series.

Capability: parity

Sixteen for sixteen. Three models you can download, inspect and self-host matched a proprietary flagship on every task — including finding a bug that only surfaces when one interval is fully contained in another, and counting characters, which language models have historically been poor at.

For ordinary, verifiable, one-shot work, the open-weight question is settled: capability is no longer the reason to choose closed.

Cost: not what the narrative says

Two things here contradict the usual framing.

GPT-OSS 120B is 18.8× cheaper than the closed flagship for identical results — and it did that while being the second most verbose model in the table. At $0.15/$0.60 per million tokens, verbosity barely registers.

Kimi K3 costs 42% more than the closed model it matches. Its per-token rate is lower than Sol's ($3/$15 against $5/$30), but it emitted 1,952 output tokens to Sol's 685. Verbosity, not rate, decides the bill — and it's the fifth time this month that the rate card pointed the wrong way.

"Open weights" is a licensing and control property. It is not a pricing tier, and treating it as one will cost you money in either direction.

Latency favours closed here

Sol returned in 15.4 seconds. GLM 5.2 and Kimi K3 took roughly 40. Only GPT-OSS 120B was faster than the closed model, at 9.2 seconds.

That's a serving characteristic more than a model one — the same open weights run 1.8–2.5× faster on one host than another, so a different provider would move these numbers substantially. Which is itself the point: with open weights you inherit a host decision that closed models make for you, and it's worth as much as most model choices.

What we'd actually do

Default to GPT-OSS 120B for verifiable one-shot work. Same results as the closed flagship, 18.8× cheaper, and faster. On a million calls shaped like this battery, that's roughly $1,200 against $21,900.

Don't assume open means cheap — measure. Kimi K3 is the counterexample in this very table. Its 1M context window and open licence may still be worth it for your use case; just don't expect a discount.

Budget for the host decision. It affects latency by up to 2.5× at identical cost and is invisible on every model leaderboard.

Keep closed models for what our battery can't measure. Everything here is single-turn and finishes in seconds. Long-horizon agentic work, sustained tool-use chains, multi-hour coding sessions — that's where the frontier benchmark leads live, and we have no data on it. If that's your workload, this comparison doesn't answer your question.

And weigh the non-benchmark factors properly. Self-hosting, data residency, model permanence (Fireworks retired five of six models we had configured earlier this year), licence terms, and the ability to pin a version indefinitely. Those are often the real reason to choose open weights, and none of them show up in a table like this one.

What changed since April

Our April comparison found closed models ahead on the harder tasks. This one finds no capability gap at all on a battery specifically designed to separate models after an easier one stopped discriminating.

That's the single largest shift we've measured this year. The open-weight question has moved from "is it good enough?" to "which host, and is it actually cheaper?" — and the second question has a less obvious answer than most people assume.

Run it yourself

• The interval-merge bug — no account, no API key

To compare open-weight and closed models on your own prompts with real token counts, that's Promptster.

Run 2026-08-11 via the Promptster API, max_tokens: 4000, one trial per model per task, all in the same session. Open-weight models served via Fireworks (gpt-oss-120b, glm-5p2, kimi-k3); GPT-5.6 Sol via OpenAI. Costs computed from list pricing against actual token usage. Single trials are enough for pass/fail against deterministic answers but not to characterise variance — we've measured 26% run-to-run cost variance elsewhere, so treat the GLM-vs-Sol cost gap as firm and any narrower gap as unresolved. Latency reflects one host; the same weights vary 1.8–2.5× across providers.