Setting max_tokens Correctly (and Why It's a Cost Control)

max_tokens looks like a minor API parameter and is actually one of the most important cost controls you have. Getting max_tokens best practices right caps the most expensive part of every call - the output - at the source. Getting it wrong is the single most common LLM cost bug we see.

Why it's a cost control, not just a limit

max_tokens is a hard ceiling on how many tokens the model may generate. Since output tokens cost several times more than input, an unbounded (or generous) max_tokens is a blank cheque: on any call, the model *could* run to the ceiling, and occasionally it will.

How to pick the number

1. 01Measure real outputs. Look at the output-token distribution for the feature. Set the ceiling a comfortable margin above the 95th percentile, not above the theoretical maximum.
2. 02Set it per feature. A classifier needs 10 tokens; a report generator needs thousands. One global value is wrong for both.
3. 03Handle the truncation case. If a response hits the ceiling (stop_reason of max_tokens/length), detect it and continue or degrade - don't ship a half-sentence.

const res = await guard.messages.create({
  model: "claude-sonnet-5",
  max_tokens: 512,   // sized to this feature's real p95, not the max
  messages,
});

if (res.stop_reason === "max_tokens") {
  // hit the ceiling - continue, summarise, or flag rather than truncate silently
}

Common mistakes

• One value everywhere. Copy-pasting max_tokens: 4096 into every call over-provisions the cheap ones and may starve the expensive ones.
• Confusing it with input cost. max_tokens only bounds output. Input is bounded by trimming history - see token usage limits.
• Ignoring it in agents. In a loop, an oversized max_tokens multiplies across steps and feeds compounding cost.

max_tokens is the per-call floor of a layered strategy: cap output here, cap input by trimming, and cap the total with a hard spend cap per user. The smallest setting that doesn't truncate real answers is the right one.