Best LLM for JSON Extraction, October 2026: 25 Configurations, 125 Strict Parses, a 100x Cost Spread

In August, four models went 36 for 36 on JSON extraction, and we concluded you should pick on cost. For October we made the ticket harder, ran it on 25 configurations instead of 4, and graded it strictly: the whole response must parse as JSON, with no code fences and no extra text.

All 125 responses passed. No model wrapped its answer in a code fence, added a preamble, picked the decoy order id, used the old email, or got the refund amount wrong. The differences are cost, which varied about 100×, latency, which varied about 30×, and some byte-level formatting that matters if you compare raw strings.

The task

Return ONLY a JSON object (no code fences, no other text) with keys:
order_id (string), reply_email (string), refund_amount (number, USD),
priority ("low", "normal" or "high"; "high" if a chargeback is threatened).

The ticket contains several traps. There are two order ids (ORD-77412 is the right one, ORD-77104 "was the toaster"), an old email the customer says to ignore, $89.50 incl shipping to convert to a number, and "i dispute the charge w/ my bank", which should map to "high". Correct answer: ORD-77412, m.okafor@example.net, 89.5, "high".

How we tested

This is one of three script-graded tasks in our October battery, alongside a four-step arithmetic problem and a three-bullet instruction-adherence task. Five runs per task, temperature 0.7 where the model accepts one, max_tokens 1500 (900 for Groq's Qwen 3.8 27B), run on 2026-10-19 through our own provider handlers. Prices are list rates from shared/pricing.ts. Every configuration passed all 15 of its calls across the three tasks, so this post is about cost, speed and formatting, not accuracy. The grader also records a lenient diagnostic that strips code fences before parsing. It never had anything to strip.

Results, JSON task only

Median of five runs, sorted by cost. "Per 1,000" is the median call cost × 1,000. For Gemini and Grok this includes the reasoning tokens those providers report outside the output count, and Grok's cached prompt prefix is priced at xAI's cached rate. Promptster's displayed cost left reasoning out until 2026-10-22 (details); where the old figure differs, it is in brackets.

*Perplexity's own billed figure, which includes a per-request fee. See below.

Together's Qwen3-235B (HTTP 400, now dedicated-endpoint only), both Cerebras models (HTTP 402, our key was unfunded) and Mistral Small (HTTP 429) did not run.

Finding 1: on this task, cost is the decision

The cheapest and most expensive configurations returned the same four values every time. Claude Fable 5.1 cost 101× more than Claude Haiku 5.5 for identical results, and GPT-6 Luna matched Haiku to within a cent per thousand calls. At a million extractions a month that is roughly $47 on Haiku 5.5 against $4,740 on Fable 5.1. For work shaped like this ticket, a flagship buys nothing you can measure.

Finding 2: some models think before writing 100 characters

The JSON object is about 100 characters. Output tokens for it ranged from 30 (Sonar Pro) and 33 (every GPT-6 Astra, GPT-6 Sol, GPT-6.1 Sol and GPT-5.6 Sol run, with zero reasoning) to 245 for Kimi K3, of which a median of 198 were reasoning. Gemini 3.6 Flash wrote 46 visible tokens and spent a median of 621 thinking tokens first.

You pay for that reasoning, and on this task it did not improve accuracy: the models with zero reasoning tokens passed 5 of 5 as well. That is why Gemini 3.6 Flash, which our August post ranked cheapest for extraction, lands in the more expensive half of this table. August's figures came from Promptster's cost calculation, which did not count thinking tokens at the time. On this ticket the corrected Gemini 3.6 Flash cost is 9.2× the displayed one, so read the August ranking with that in mind. Promptster's display has counted thinking tokens since 2026-10-22.

Finding 3: "valid JSON" isn't "the same bytes"

Every response parsed to the same object. The raw strings were not identical. Three shapes from the run files:

{"order_id":"ORD-77412","reply_email":"m.okafor@example.net","refund_amount":89.50,"priority":"high"}
{"order_id": "ORD-77412", "reply_email": "m.okafor@example.net", "refund_amount": 89.50, "priority": "high"}
{
  "order_id": "ORD-77412",
  ...
}

These are GPT-6 Sol (compact), Claude Opus 5.5 (spaced) and DeepSeek V4 Pro (pretty-printed). Three details that come up in production:

• Pretty-printing. DeepSeek V4 Pro and Groq's Qwen 3.8 27B always returned multi-line JSON. gpt-oss-120b did in 3 of 5 runs. A line-based parser, a log pipeline that expects one record per line, or a regex for ^\{.*\}$ would break on these.
• 89.50 vs 89.5. Both are valid JSON numbers and parse to the same value. Eleven configurations, including GPT-6 Sol, Sonnet 5.5, Opus 5.5 and both Kimi K3 hosts, copied the ticket's 89.50 on every run. gpt-oss-120b and Sonar Pro wrote 89.5 every time. If you hash raw output for caching or compare it in snapshot tests, these are different answers.
• Mixed formatting from the same model. Kimi K3 and both Groks switched between compact and spaced from run to run. Even at a fixed temperature, don't assume a model formats the same way twice.

The fix is the same in all three cases. Parse first, then compare the parsed values. Never compare the raw strings.

Finding 4: code fences didn't appear, but don't drop the guard

We expected some code fences, which break JSON.parse on otherwise correct output. There were none in 125 responses. Our prompt says "no code fences" explicitly, and we did not test one without that instruction, so this shows the instruction works, not that the guard is unnecessary. Keep the fence-stripping fallback.

A note on Sonar Pro's cost

Perplexity's usage object includes its own cost breakdown. On the first JSON run it reported input_tokens_cost $0.00162, output_tokens_cost $0.00018 and request_cost $0.0025, for a total_cost of $0.0043. Our list-rate calculation gave $0.000855, because our pricing had no per-request fee. Since 2026-10-22 Promptster displays Perplexity's own total_cost, which is the figure in the table: $0.00417–$0.0043 across the five runs, a median of $4.18 per thousand calls.

What we'd use

• Lowest cost: Claude Haiku 5.5 or GPT-6 Luna, about $0.05 per thousand.
• Lowest latency: Qwen 3.8 27B on Groq at 0.21 s, or gpt-oss-120b on Groq at 0.68 s for less than half the price. Both pretty-print, so parse properly.
• Fast and compact from a closed provider: GPT-6 Sol or Claude Sonnet 5.5, around 1.1–1.2 s.

Try it yourself

• Run the JSON extraction task — check whether you get 89.50 or 89.5

No account and no API key needed. To run your own extraction schema across models and see output tokens and cost per call, use Promptster.

Run 2026-10-19 through Promptster's provider handlers: 25 configurations that returned results, five runs of the JSON task each, temperature 0.7 where accepted (OpenAI GPT-5+ and Claude 5+ drop it), max_tokens 1500 (Groq Qwen 3.8 27B at 900). Strict grading: the whole response must parse with JSON.parse and all four keys must match exactly. Costs are list rates from shared/pricing.ts. Gemini and Grok costs price reported reasoning tokens at the output rate. Grok and Sonar Pro use the provider's reported cost, which includes xAI's cache discount and Perplexity's request fee. Latency is wall-clock from one machine. Raw responses and usage objects are in content/blog/data/q4-2026-battery/runs/.