Together vs Fireworks: The Same Open Weights Run 1.8-2.5x Faster on One Host
By Promptster Team · 2026-08-28
When you pick an open-weight model you make two decisions, and most write-ups only cover the first. The model is the obvious one. The host is the one nobody benchmarks — and it turns out to matter more than several model choices we've measured this month.
Together and Fireworks both serve Kimi K3 at an identical $3/$15 per million tokens, and both serve GPT-OSS 120B at an identical $0.15/$0.60. Same weights, same price, two providers. Every difference is infrastructure.
We ran five trials of each combination.
Fireworks won every comparison
| Configuration | Median latency | All five trials (ms) | Median tok/s | Avg cost |
|---|---|---|---|---|
| Kimi K3 — Fireworks | 23,181 ms | 23181, 20628, 24858, 25110, 22659 | 51.8 | $0.018348 |
| Kimi K3 — Together | 41,929 ms | 35310, 41929, 24321, 45949, 44795 | 28.6 | $0.018339 |
| GPT-OSS 120B — Fireworks | 2,946 ms | 2360, 2946, 2664, 4996, 2956 | 196.9 | $0.000372 |
| GPT-OSS 120B — Together | 7,279 ms | 7279, 11874, 7338, 4614, 5615 | 90.9 | $0.000403 |
Kimi K3: Fireworks 1.81× faster. GPT-OSS 120B: Fireworks 2.47× faster.
Costs are effectively identical — the small differences are token-count variation between runs, since the per-token rates match exactly.
Two models, same direction, so this isn't a quirk of one deployment. It also brackets the 2.00× we measured on a single run earlier this month, which is reassuring: the one-run figure landed between the two properly sampled ones.
Variance is the underreported half
Look at the trial columns rather than the medians.
Kimi K3 on Together ranged 24,321 to 45,949 ms — the slowest run took 1.9× the fastest. On Fireworks the same model ranged 20,628 to 25,110 ms, a 1.2× spread.
GPT-OSS 120B on Together ranged 4,614 to 11,874 ms, a 2.6× spread. On Fireworks, 2,360 to 4,996 ms.
Median latency tells you what a typical request costs in time. Spread tells you what your p95 looks like, and that's what your users actually feel. On both models Together was not just slower but substantially less predictable — which for an interactive product is the more expensive property of the two.
Throughput follows
Fireworks produced 51.8 tok/s against 28.6 on Kimi K3, and 196.9 against 90.9 on GPT-OSS 120B — roughly 1.8× and 2.2×. Since the weights are identical, that's a serving-stack difference: batching strategy, hardware, quantisation choices, queue depth at the moment we tested.
What this does and doesn't establish
It's a snapshot. Twenty requests from one region on one afternoon. Inference providers tune continuously and load varies by hour; a re-run next month could differ. The finding to carry forward is "measure your host, the gap is large enough to matter" — not "Fireworks is permanently 2× faster."
We didn't test under concurrency. Serving stacks differentiate most under parallel load, which is where batching strategy really shows. Single sequential requests are the easy case.
We didn't test everything that decides procurement. Rate limits, model availability, cold starts, regional endpoints, uptime, support, data-handling terms. Fireworks retired five of the six models we had configured earlier this year with no notice we caught, which is its own kind of operational cost.
Cost was a wash here precisely because we chose models with matching list prices. That's what made the comparison clean; it isn't true across their whole catalogues.
The practical read
Benchmark the host, not just the model. A 1.8–2.5× latency difference at identical cost is larger than most model-to-model differences we measured this month. If you've selected an open-weight model and defaulted to whichever host you already had an account with, that's an unexamined decision worth an afternoon.
Test your own model. The effect differed between the two models we tried (1.81× vs 2.47×), so it isn't a fixed provider constant.
Measure spread, not just the mean. Together's p95 behaviour was the bigger practical gap. Run at least five trials and look at the range.
Re-check periodically. This is infrastructure, and infrastructure changes without a changelog.
Run it yourself
- The prompt we used — no account, no API key
To run the same model on both hosts and compare latency and throughput directly, that's Promptster.
Run 2026-08-11 via the Promptster API, max_tokens: 1200, five trials per configuration, all from a single region in one session so network conditions are shared. Kimi K3 tested as moonshotai/Kimi-K3 (Together) and accounts/fireworks/models/kimi-k3 (Fireworks), both listed at $3/$15 per million input/output tokens. GPT-OSS 120B as openai/gpt-oss-120b and accounts/fireworks/models/gpt-oss-120b, both at $0.15/$0.60. Latency is end-to-end including network round-trip. Medians are reported alongside every individual trial so you can judge the spread yourself.