Open-Weight vs Closed-Source, October 2026: The Cheapest Calls Were Closed, the Fastest Were Open

This is the October refresh of Benchmarking Open-Source vs. Closed-Source Models in 2026, our most-read comparison, and of the August follow-up. In April the common view was that open weights were the cheap option and closed models were the quality option. In October, neither half of that held up in our data.

Every configuration we could run passed all 15 graded calls. So the differences below are cost, latency and token usage, not accuracy. Within those differences:

• The two cheapest configurations were closed: GPT-6 Luna and Claude Haiku 5.5.
• The two fastest were open-weight, both hosted on Groq.
• Kimi K3, the open model most often set against the flagships, cost more than Claude Opus 5.5 for the same 15 calls.

How we tested

Three script-graded tasks: strict JSON extraction from a messy support ticket, a four-step arithmetic word problem (answer: 580), and an instruction-adherence task (exactly three bullets, each under 12 words, no preamble). Five runs per task, so 15 calls per configuration. Temperature 0.7 where the model accepts one, max_tokens 1500 (900 for Groq's Qwen 3.8 27B, because of an output-token rate limit on Groq's on-demand tier), run on 2026-10-19 through our own provider handlers. Prices are list rates from shared/pricing.ts.

Open-weight: Kimi K3 (on both Together and Fireworks), GLM 5.3 and DeepSeek V4.1 Flash (Fireworks), gpt-oss-120b and Qwen 3.8 27B (Groq). Closed: the GPT-6 family, Claude 5.5 family plus Fable 5.1, Gemini 4 Argon and 3.x Flash, Grok 4.7 and 4.5.

The table

Total cost for all 15 calls, sorted cheapest first. For Gemini and Grok it counts the reasoning tokens those providers report outside the output count, and for Grok it prices xAI's cached prompt prefix at the cached rate. Promptster's displayed cost has done both since 2026-10-22 (why).

Median latency is across all 15 calls, measured from one machine, and is not a host SLA.

Finding 1: open weights are no longer the cheap option

In April, the case for open weights started with price: "3-5x cheaper per prompt." That is no longer true at the bottom of the market. GPT-6 Luna and Claude Haiku 5.5 both list at $0.10 / $0.50, below every open-weight configuration in this test. On our battery, Luna cost 2.7× less than gpt-oss-120b on Groq, the cheapest open configuration, and Haiku cost 2.0× less.

Open weights do still undercut the closed mid-tier. gpt-oss-120b ran the battery for about a fifth of GPT-6 Sol's cost ($0.0029 against $0.0144), and DeepSeek V4.1 Flash for well under half. In August, gpt-oss-120b beat GPT-5.6 Sol by 18.8×. Against GPT-6 Sol, which OpenAI prices at half of GPT-5.6 Sol, the gap is 4.9×.

Finding 2: the fastest configurations were open, because of the host

Qwen 3.8 27B on Groq had a median of 0.2 s on two of the three tasks and 1.2 s on the arithmetic problem, the fastest in the battery. gpt-oss-120b on Groq was second at 0.9 s overall. The fastest closed configuration was Claude Haiku 5.5 at 1.0 s.

That is a hosting result more than an open-weight result. The same Kimi K3 weights took 5.0 s on Together and 7.0 s on Fireworks. We would have liked a Cerebras row as a second fast host, but our Cerebras key was unfunded during this run (HTTP 402), so there isn't one. Our next post looks at the host variable on its own.

Finding 3: Kimi K3 costs more than Opus 5.5

Kimi K3's list rate ($3 / $15) is below Claude Opus 5.5's ($4 / $20). Across our 15 calls it still cost 14% more: $0.0715 against $0.0628.

The difference is output length. On the JSON task, where the correct answer is about 100 characters, Kimi K3 on Together used a median of 245 output tokens, 198 of them reasoning. Opus 5.5 used 53, with no thinking tokens reported. On the three-bullet task Kimi used 310 to Opus's 189. A lower rate per token does not help if the model produces several times as many tokens. In August, Kimi K3 cost 42% more than its closed comparison. The gap is smaller now, but it points the same way.

What didn't run

• Together's Qwen3-235B returned HTTP 400 model_not_available on every call: "Unable to access non-serverless model … create and start a new dedicated endpoint." The model is still in our registry, but Together now serves it only on dedicated endpoints, so a pay-per-token key cannot reach it. We have flagged the registry entry.
• Cerebras (gpt-oss-120b, Qwen 3.8 27B): HTTP 402, unfunded key.
• Mistral Small: HTTP 429 on the first call. Not in this comparison.

What this battery can't tell you

Our three tasks are single-turn and finish in seconds. Every model passed every one, so this data cannot rank models on capability. It also does not measure long-context work, multi-step agents or tool use, which is where the expensive closed models are positioned. The April post found a reasoning edge for closed models using an LLM judge. Our arithmetic task was graded by script, and every model got 580 on every run. That does not show the edge has disappeared. It shows our task is too easy to detect it.

What we'd do

At the low end, choose on cost and stop treating "open" as a synonym for cheap. For short, verifiable tasks like these, GPT-6 Luna and Claude Haiku 5.5 are the cheapest options we measured.

If latency is the constraint, open weights on a fast host are the answer. Groq's Qwen 3.8 27B and gpt-oss-120b were the only configurations under one second.

Choose open weights for the reasons price lists don't show: you can move the same model to another host, run it yourself, or fine-tune it. Those benefits are real. Lower cost is no longer automatic.

Measure output tokens, not list rates. Kimi K3 shows why: a lower rate per token still cost more than Opus 5.5 because of how much it wrote.

Try it yourself

• Run the three-bullet task — every configuration passed it; compare how many tokens each one spends

No account and no API key needed. To put open-weight and closed models side by side on your own prompts, with token counts and cost, use Promptster.

Run 2026-10-19 through Promptster's provider handlers: 19 configurations, three script-graded tasks, five runs each, temperature 0.7 where accepted (OpenAI GPT-5+ and Claude 5+ drop it), max_tokens 1500 (Groq Qwen 3.8 27B at 900). Costs are list rates from shared/pricing.ts with reasoning tokens priced as output, xAI's reported cost (which applies its cache discount) for Grok, and no off-peak or promotional adjustments beyond what that file lists. Gemini 3.x Flash rates are promotional per that file. Raw responses and usage objects for every call are in content/blog/data/q4-2026-battery/.