Same Open-Weight Model, Different Host, October 2026: Together Now Beats Fireworks on Kimi K3

Open weights let you choose where the model runs. We ran the same Kimi K3 weights on Together and on Fireworks, at the same list price, through the same three graded tasks.

Both hosts passed all 15 calls, and the bills matched to four decimal places: $0.0715 each. The only difference was latency, and it has reversed since August. In August, Fireworks served Kimi K3 2× faster and won every latency comparison against Together. In October, Together had the faster median on all three tasks. Fireworks had the more predictable latency.

We also ran gpt-oss-120b and Qwen 3.8 27B on Groq. The second host for those, Cerebras, could not be tested: our Cerebras key was unfunded during this run.

How we tested

Three script-graded tasks: strict JSON extraction from a messy support ticket, a four-step arithmetic word problem (answer: 580), and an instruction-adherence task (exactly three bullets, each under 12 words, no preamble). Five runs per task per host, temperature 0.7, max_tokens 1500 (900 for Qwen 3.8 27B on Groq, see below), run on 2026-10-19 through our own provider handlers. Prices are list rates from shared/pricing.ts. Latency is wall-clock from one machine, not a host SLA, and five runs is a small sample. We also sent a ten-word probe prompt three times per host.

Every configuration that ran passed all 15 calls. There was no accuracy difference between hosts.

Kimi K3: Together vs Fireworks

Together's median was 1.3–1.6× faster on the graded tasks and 3.1× faster on the probe. Fireworks never took longer than 10 seconds. Together took 23.6 s on one arithmetic run and 17.0 s on one three-bullet run, while most runs of the same prompts finished in 4–6 s.

Which host is better depends on what you are optimising. For a batch job, where only the median matters, Together came out ahead in this run. For an interactive feature with a timeout, Fireworks' tighter spread matters more than its slower median. A 23-second outlier is the call that hits your timeout and gets retried.

The output was the same

The weights are the same, and so was the behaviour. Median output tokens per task were 245 vs 242 (JSON), 306 vs 302 (arithmetic) and 310 vs 320 (bullets), with reasoning tokens almost identical: 198 vs 197 on the JSON task. Prompt tokens differed by exactly 3 on every task (97 vs 100 on the probe), which is a difference in chat templates between the hosts, not in the model.

That is why the costs match: same price, same token counts. You choose between these two hosts on latency, nothing else.

What changed since August

We don't know. Our August and October runs used different batteries and different harnesses (August used our hosted API, October calls the provider handlers directly), so the two results are not a controlled before-and-after. What we can say is that a two-month-old latency comparison between hosts no longer described what we measured. If host speed matters to your product, measure it on a schedule rather than once.

gpt-oss-120b and Qwen 3.8 27B: Groq only

These were the two fastest configurations in our whole battery of closed and open models. Qwen 3.8 27B returned the JSON object and the three bullets in about 0.2 seconds.

Cerebras, the obvious second host, returned HTTP 402 payment_required on the first call for both models. Our Cerebras key was unfunded during this run, so we have no Cerebras latency or cost to report, and we have not estimated them. The list prices above are from our registry, not measured.

The list prices do suggest where the comparison will land. On Qwen 3.8 27B, Cerebras charges more for input ($0.99 vs $0.80) and much less for output ($1.49 vs $4.00). Our arithmetic task produced a median of 497 output tokens on Groq, so it is output-heavy. If Cerebras produced the same token counts, which we have not verified, that task would cost about $0.0009 per call there, against $0.0021 measured on Groq.

The Groq rate limit

On Groq's on-demand tier, Qwen 3.8 27B is capped at 1,000 output tokens per minute. A request with max_tokens 1500 was rejected with HTTP 429 before generating anything, so we ran it at max_tokens 900. No response was truncated, and every call finished with stop.

The cap still showed up. 4 of the 15 calls got a 429 on the first attempt and succeeded when retried 30 seconds later. The latency figures above are for the successful attempt and do not include the wait. At about 500 output tokens per arithmetic answer, a 1,000-token-per-minute budget allows roughly two such answers a minute. The 0.2-second latency is real, but on this tier you cannot make many calls in a row.

gpt-oss-120b on Groq had no rate-limit retries in our run.

What we'd do

Choose the host after the model, and measure it. Identical weights and prices still produced a 3× gap on the probe and very different worst cases. That holds whichever host leads in a given month.

Look at the slowest calls, not just the median. Together's median was better. Fireworks' worst case was less than half of Together's. Which of those matters is a product decision.

Check rate limits before latency. Groq's Qwen 3.8 27B was the fastest model we measured and also the easiest to rate-limit. On a free or on-demand tier, throughput can be a tighter limit than speed.

We'll add Cerebras once the key is funded. Until then, this comparison is Groq-only for those two models, and we'd rather say that than estimate the missing numbers.

For the wider picture on open weights versus closed models, see our October comparison and the original open-source vs closed-source benchmark.

Try it yourself

• Run the three-bullet task — Qwen 3.8 27B on Groq answered it in about 0.2 s; Kimi K3 took 6–8 s

No account and no API key needed. To run the same open-weight model on several hosts and compare latency on your own prompts, use Promptster.

Run 2026-10-19 through Promptster's provider handlers: Kimi K3 on Together (moonshotai/Kimi-K3) and Fireworks (accounts/fireworks/models/kimi-k3), gpt-oss-120b and Qwen 3.8 27B on Groq; three script-graded tasks, five runs each, plus a ten-word probe run three times. Temperature 0.7, max_tokens 1500 (Qwen 3.8 27B on Groq at 900). Cerebras returned HTTP 402 for both of its models and is not measured. Costs are list rates from shared/pricing.ts. Latency excludes the 30-second wait before any 429 retry. Raw responses and usage objects for every call are in content/blog/data/q4-2026-battery/ and content/blog/data/q4-2026-token-overhead/.