Why Is My LLM Response Cut Off Mid-Sentence? Thinking Tokens Spent the max_tokens Budget
By Promptster Team · 2026-11-10
Your model starts a sensible answer and stops halfway through a line. The request returned HTTP 200 and no error, and you were billed for the full token limit.
Usually the cause is that the model spent most of your max_tokens thinking before it started to write. On some providers, max_tokens covers hidden reasoning as well as the visible answer. When we capped a ten-step arithmetic task at 1,500 tokens, Gemini 3.8 Flash and Gemini 3.6 Flash went from 5/5 to 0/5. Every response was cut off, and in each one thinking had used 1,218 to 1,440 of the 1,500 tokens. At 4,000 tokens, both passed every run. Eight other models handled the same 1,500 cap without trouble.
This is the partial-answer version of a problem we covered in Why Your LLM Returns an Empty Response. Cause 3 in that post is the same budget running out before the model writes anything at all.
The test
The task is stock-floor. A warehouse starts with 1,200 units, ships 15% each weekday rounded down, and receives 140 units on Wednesdays and Fridays. The question is the stock at the end of the second Friday. The answer is 587, and the prompt asks for brief working followed by a final Answer: <number> line. The grader checks the final number, with no LLM judge.
We ran it five times per configuration (n=5) on 2026-10-23 through Promptster's provider handlers: once in the main battery at max_tokens 4000, and again as a separate data set at max_tokens 1500. The Gemini handler sends max_tokens as maxOutputTokens and sends no thinking configuration, so each model used its default thinking. Costs are cost_usd at list rates from shared/pricing.ts.
What the cap did
| Config | Passed at 1500 | Truncated at 1500 | Median reasoning tokens at 1500 | Median reported output tokens at 1500 | Passed at 4000 |
|---|---|---|---|---|---|
| Gemini 3.8 Flash | 0/5 | 5 (MAX_TOKENS) |
1,326 | 170 | 5/5 |
| Gemini 3.6 Flash | 0/5 | 5 (MAX_TOKENS) |
1,438 | 57 | 5/5 |
| Gemini 4 Argon | 5/5 | 0 | none reported | 745 | 5/5 |
| GPT-6 Luna | 5/5 | 0 | 179 | 369 | 5/5 |
| GPT-6 Sol | 5/5 | 0 | 188 | 301 | 5/5 |
| Claude Haiku 5.5 | 5/5 | 0 | — | 757 | 5/5 |
| Claude Sonnet 5.5 | 5/5 | 0 | — | 569 | 5/5 |
| DeepSeek Flash | 5/5 | 0 | 547 | 697 | 5/5 |
| gpt-oss-120b (Groq) | 5/5 | 0 | 857 | 1,153 | 5/5 |
| Grok 4.7 | 5/5 | 0 | 1,070 | 188 | 5/5 |
The output column means different things on different providers. On the OpenAI-compatible APIs (GPT-6, DeepSeek, Groq), reported output tokens include reasoning. Gemini reports the visible answer as candidatesTokenCount and the thinking separately as thoughtsTokenCount. Grok 4.7 also reports its reasoning separately from the visible output.
What a cut-off response looks like
This is how the visible output of one Gemini 3.8 Flash run at 1500 ends, after the opening line and the first two days:
- Wednesday: Starts with 867. Ships $\lfloor 867 \times 0.15 \rfloor = 130$. Receives 1
The response ends at "Receives 1". Here is its usage object, verbatim:
{"promptTokenCount": 102, "candidatesTokenCount": 170, "totalTokenCount": 1598, "promptTokensDetails": [{"modality": "TEXT", "tokenCount": 102}], "thoughtsTokenCount": 1326, "serviceTier": "standard"}
170 visible tokens plus 1,326 thinking tokens comes to 1,496, which is effectively the whole 1,500. finishReason was MAX_TOKENS.
Gemini 3.6 Flash ran out sooner, with 56 to 94 visible tokens per run. In one run the visible text was the model restating the prompt in quotes, and it stopped partway through:
"Nothing happens on weekends." "How many units
At 4000, the same models used more thinking than they managed under the cap (median 1,415 vs 1,326 for 3.8 Flash, and 1,877 vs 1,438 for 3.6 Flash). Every one of Gemini 3.6 Flash's runs at 4000 used between 1,789 and 2,272 thinking tokens, more than the entire 1,500 budget. At a 1500 cap, that model could not have finished this task.
You pay for the truncated answer
Each truncated Gemini Flash call cost $0.0057, priced as about 1,496 output-rate tokens. The five capped Gemini 3.8 Flash calls cost $0.0284 in total and produced no correct answers. The five uncapped calls cost $0.0361 and passed 5/5. A lower cap cut the bill a little and made every response unusable.
Not only Gemini: a host cap
Qwen 3.8 27B on Groq ran with max_tokens 900 in the main battery. Groq applies the on-demand tier cap of 1,000 output tokens per minute to this model, so a larger request is rejected with HTTP 429 before it generates anything. All five stock-floor runs stopped at exactly 900 output tokens with finish_reason: length, partway through the working:
- End: 617 – 92 = 525
Friday (Week 2)
- Start:
Groq reported no reasoning tokens for this model, so the 900 tokens were all visible working. The model was simply too verbose for the cap, and we could not raise the cap on that tier. That stock-floor 0/5 is a measurement of the limit, not of the model's arithmetic.
How to tell, in 30 seconds
Check the stop field before you read the text.
| Provider format | Stop field | Truncated value | Where the thinking is counted |
|---|---|---|---|
| OpenAI-compatible (OpenAI, DeepSeek, Groq, Together, Fireworks…) | choices[0].finish_reason |
length |
usage.completion_tokens_details.reasoning_tokens, included in completion_tokens |
| Google Gemini | candidates[0].finishReason |
MAX_TOKENS |
usageMetadata.thoughtsTokenCount, not included in candidatesTokenCount |
| Anthropic | stop_reason |
max_tokens |
inside output_tokens |
If the stop value is the truncated one, do not parse the answer. A last-number grader would have read "1" from "Receives 1". A naive Answer: regex finds nothing and might return a default.
In this data, truncation also showed up as the total being close to the cap. If visible output plus reasoning sits within a few tokens of max_tokens, the model ran out of budget; it did not decide to stop. The hidden-tokens post has more on reading these fields.
What to change
Treat max_tokens as a budget for thinking and the answer together. For multi-step tasks on models that think by default, 1,500 was too low here. 4,000 was enough for every config that ran at 4,000.
Check the stop field on every call, not just in tests. A truncated answer looks almost like a real one. Log finish_reason/finishReason, and retry with a larger cap or treat the call as failed.
If cost is why you lowered the cap, lower the thinking instead. Cutting the cap still bills the thinking and loses the answer. Where a model lets you set its thinking level (Gemini through its thinking configuration), use that.
Check per-tier output limits on your host. The on-demand tier cap of 1,000 output tokens per minute on Groq set our Qwen limit. No setting on our side could have fixed that run.
Run this comparison yourself — no account, no API key
- Run the stock-floor task: the correct answer is 587. Check the token counts and the stop reason as well as the number.
To run it across models with different max_tokens settings and see the finish reason and token breakdown for each, use Promptster. For how the same models compare on cost and quality more broadly, see the original open-source vs closed-source benchmark.
Run 2026-10-23 through Promptster's provider handlers. Main battery: four tasks, five runs each, max_tokens 4000. Cap data set: stock-floor only, five runs each on ten configurations, max_tokens 1500. Groq Qwen 3.8 27B ran at 900 in the main battery because of the on-demand tier cap of 1,000 output tokens per minute on Groq. Temperature 0.7 where the model accepts one. Script-graded (answer 587). Costs are cost_usd at list rates from shared/pricing.ts, with Gemini thinking tokens priced as output; the Gemini Flash rates are promotional per that file. Raw responses, finish_reason and raw_usage for every call are in content/blog/data/q4b-2026-maxtokens-1500/ and content/blog/data/q4b-2026-battery/.