Claude Haiku 5.5 vs GPT-6 Luna on Harder Tasks: Luna 20/20 for $0.0019, Haiku 15/20 for $0.0036

By Promptster Team · 2026-11-16

Which $0.10 model should you pick, Claude Haiku 5.5 or GPT-6 Luna? Both list at $0.10 per million input tokens and $0.50 per million output in shared/pricing.ts. In October, on an easier battery, both passed 15/15, Luna was cheaper and Haiku was faster. On our harder November battery the tie is gone: GPT-6 Luna passed 20/20 for $0.0019; Claude Haiku 5.5 passed 15/20 for $0.0036.

All five of Haiku's misses are on one task: three taglines of exactly seven words each, with five other rules, and "Output only the three lines". Haiku passed the other three tasks 15/15. On this battery, pick Luna, unless your prompts look nothing like the tagline task and a fraction of a second per call matters to you.

Results

Four script-graded tasks, five runs each:

Task Claude Haiku 5.5 GPT-6 Luna
json-order: strict JSON with a null PO, a cancelled line, a per-item discount 5/5 5/5
stock-floor: ten-step round-down arithmetic, answer 587 5/5 5/5
seven-word-taglines: exactly 7 words, lowercase, banned and required words 0/5 5/5
launch-notes: final date and budget from superseded notes 5/5 5/5
Total 15/20 20/20
Spend, all 20 calls $0.0036 $0.0019

Haiku got the hard parts right. It never used the tracking ref as a PO, applied the discount only to the $120 item, rounded down at all ten steps, and picked the final approved budget over the superseded and unapproved ones. Every format diagnostic on those three tasks passed for both models: no code fences, a literal Answer: line, the bare Nov 21, $37500.

What the tagline task demands

The 284-character prompt asks for three taglines for a reusable drinks flask, then sets the rules:

Rules: one tagline per line; each exactly 7 words; all lowercase; no punctuation, digits, numbering or bullets; never use the words "water", "bottle" or "eco"; every tagline must include the word "refill". Output only the three lines.

The grader checks every rule: exactly three non-empty lines, each only lowercase letters and single spaces, exactly seven words, no banned word (plurals included), and the word refill. Anything else in the response, such as a note, a count or a second draft, adds lines and fails. A diagnostic re-runs the check without the word count, to separate "miscounted" from "broke another rule".

Haiku's diagnostic was 1/5, so only one of its five failures was purely a miscount. The other four broke something besides the count.

How Haiku failed

Read from the five run files:

Run What happened Diagnostic
1 Three lines, then "Let me check the counts", a correction, a second check, a third set fail
2 Three clean lines; the first, "refill it once and keep going", has six words pass
3 Three lines, a check, a revision, a word-by-word count, "Correct versions:" and a third set fail
4 "refill it daily and carry the planet with you" (nine words); another line used "refills", not "refill" fail
5 Three lines, a check, "Corrected set below." and a revised set fail

Three of five runs answered first and then checked their work in the visible reply. Run 5, verbatim:

refill it again and again every sip starts with a refill keep your drinks cold and refill often

Let me check the counts: "refill it again and again" = 6 words. That fails. Corrected set below.

The checking did not always work. "refill it again and again" has five words, not six; Haiku made the same miscount in run 1. In run 3, its count said the first line has nine words; "refill it daily and keep your drinks cold" has eight. In run 1 the final set still contained an eight-word line, which Haiku counted correctly ("= 8") and printed anyway.

So the tagline task fails Haiku in two ways at once. Its first drafts miss the count (all five first drafts had at least one line that was not seven words: 5, 6, 8 and 9 words all appear), and when it notices, it fixes things in the output rather than before it, which breaks "Output only the three lines".

How Luna passed

Luna's five responses are three lines each, nothing else. One, verbatim:

refill your day with fresh lasting flavor carry comfort refill wherever life takes you make every journey brighter refill and repeat

The difference is where the checking happens. Luna is a reasoning model: Promptster sends it reasoning_effort: medium, and OpenAI reported 130–206 reasoning tokens per tagline call, before the visible answer. Haiku's usage object reports no separate reasoning on any of its tagline calls; its checking, when it happened, was in the reply. We did not test Haiku with extended thinking turned on, so we cannot say whether that would close the gap.

Cost and speed

Claude Haiku 5.5 GPT-6 Luna
Spend, 20 calls $0.0036 $0.0019
Spend per passed call $0.00024 $0.00009
Median input tokens (json / stock / tag / launch) 202 / 150 / 106 / 239 142 / 102 / 72 / 151
Median output tokens (json / stock / tag / launch) 230 / 726 / 99 / 192 86 / 282 / 174 / 68
Median latency, s (json / stock / tag / launch) 1.6 / 3.2 / 1.5 / 1.3 1.6 / 3.5 / 2.0 / 1.4

Haiku cost 1.9× Luna at the same rates, up from 1.36× in October. The gap is tokens. Anthropic counts the same prompts as 42–58% more input tokens, and Haiku wrote longer answers on three tasks: a median of 726 output tokens on stock-floor against Luna's 282, which includes Luna's reasoning. The one task where Haiku was cheaper was the one it failed. Its tagline calls cost $0.00041 in total against Luna's $0.00050, because the failed runs were short.

Haiku was still a little faster. Its median was lower on three tasks and within 0.03 s of Luna's on json-order, and its overall median was 1.6 s against Luna's 1.9 s. October's gap, where Haiku was roughly twice as fast on two tasks, has mostly closed.

Which to use

Method

Four script-graded tasks (no LLM judge), five runs per task per model, max_tokens 4,000, run 2026-10-23 through Promptster's own provider handlers. Neither model accepts a custom temperature, so none was sent; Luna went out with reasoning_effort: medium. cost_usd is calculateCost() from shared/pricing.ts at list rate, with no cache or batch discounts; Haiku's long-prompt tier ($0.50 / $2.50 over 100K tokens) does not apply to prompts this short. Spend per passed call is total spend divided by calls passed. Latency is wall-clock from one machine, not a provider SLA. For the open-weight models on the same battery, see Benchmarking open-source vs. closed-source models.

Run this comparison yourself — no account, no API key

To run Haiku 5.5 and Luna side by side on your own prompts, with token counts and cost per call, that's Promptster.


All figures from content/blog/data/q4b-2026-battery/ (summary.json plus one raw file per call, each provider's usage object verbatim), run 2026-10-23. Haiku's tagline responses are in runs/anthropic__claude-haiku-5-5/seven-word-taglines__run1–5.json. October comparisons are from content/blog/data/q4-2026-battery/, run at max_tokens 1,500 on different tasks.