Mistral AI Context Window Limits

Context bugs are the least dramatic failures in an LLM integration and the hardest to attribute. There is no stack trace, no non-2xx status, and often no error at all — just an answer that is subtly worse than it was last month, in a system where nobody expects the output to be deterministic in the first place. By the time someone files it, the change that caused it is dozens of deploys back.

Where the Tokens Actually Go

Most teams account for one of these and are surprised by the rest. The window is shared by everything in the list below, and the items you did not write are usually the ones that grow.

The last row is the one that causes production incidents. Input and output share the window, so a request that fits perfectly on the way in can leave the model a few hundred tokens to answer in. It will use them, stop mid-sentence, and return a 200.

Reserve the Answer First

The correct order of operations is not the intuitive one. Do not fill the window with context and then ask how much room is left for the answer — decide the answer's budget first, subtract it, and let history compete for what remains.

// Order matters. Output budget is a requirement, not a leftover.
const WINDOW = windowForModel('mistral-large-latest');
const MAX_OUTPUT = 1_500;        // what a complete answer actually needs
const SAFETY_MARGIN = Math.ceil(WINDOW * 0.05); // estimation error, chat scaffolding

const systemTokens = estimateTokens(SYSTEM_PROMPT);
const toolTokens = estimateTokens(JSON.stringify(TOOL_DEFINITIONS));

const historyBudget = WINDOW - MAX_OUTPUT - SAFETY_MARGIN - systemTokens - toolTokens;

if (historyBudget < MIN_VIABLE_HISTORY) {
  // The fixed costs no longer leave room for a conversation. That is a
  // design problem, not a runtime one — fail loudly rather than sending
  // a request that cannot produce a good answer.
  throw new ContextBudgetExhausted({ systemTokens, toolTokens, WINDOW });
}

const messages = trimToFit(history, historyBudget);

The explicit throw matters more than the arithmetic. Without it, a system prompt that grew past the point of viability degrades silently: every request still succeeds, every answer is slightly worse, and the graph that would have shown you is one nobody built.

Four Trimming Strategies and What Each One Loses

Two rules survive across all four. Never trim to a message boundary that separates a tool call from its result — most APIs reject the orphan, and the ones that do not will produce something stranger. And never summarise text the user needs reproduced verbatim: identifiers, code, exact quotes. Those get pinned or dropped, never paraphrased.

Reading a Context Failure

Rows two through four are the reason this family of bug survives so long: all three return a200 with a well-formed body. Nothing in a conventional error budget or uptime monitor will ever fire on them. The only instrument that catches them is a per-request record of input tokens, output tokens, and finish reason.

The Mistral Trap: Counting Tokens With Somebody Else’s Tokenizer

Most token-budgeting code in the wild was written against OpenAI first, which means it counts with a tiktoken encoding. That library does not know how a Mistral model segments text, and the gap is not a rounding error — it varies with the content. Code, non-English text, and heavy punctuation are exactly where the two disagree most, and exactly where prompts tend to be longest.

The failure this produces is quiet in the direction that matters. If your counter under-estimates, you build a trimmer that believes it left comfortable headroom, and it ships a request the API rejects — or worse, one the API accepts after silently dropping the part you cared about. The trimmer looks correct in every test you wrote, because your tests used English prose.

So either count with Mistral's own tokenizer, or stop pretending to count precisely and budget by characters with a deliberately pessimistic ratio. The second option is less elegant and much harder to get subtly wrong, and for a trimming heuristic that is the better trade.

Budget conservatively when you cannot count exactly:

// A tiktoken count is a guess for a Mistral model. If you are not using
// Mistral's own tokenizer, do not pretend to precision you do not have.
const PESSIMISTIC_CHARS_PER_TOKEN = 3; // deliberately low; over-reserves headroom

function estimateTokens(text) {
  return Math.ceil(text.length / PESSIMISTIC_CHARS_PER_TOKEN);
}

// Reserve room for the answer BEFORE deciding what history fits.
function budgetForHistory(windowTokens, systemPrompt, maxOutputTokens) {
  const reserved = estimateTokens(systemPrompt) + maxOutputTokens;
  const safetyMargin = Math.ceil(windowTokens * 0.05);
  return windowTokens - reserved - safetyMargin;
}

// Then trim history to fit budgetForHistory(...), oldest turns first.

When the Problem Is Not Your Budget

There is a class of incident that looks exactly like a context regression and is not one. During upstream degradation, answers get shorter and vaguer, requests time out before completing, and long prompts fail more often than short ones because they take longer to process. Every symptom points at the prompt change that shipped that morning.

The question that separates the two cases cannot be answered from your own logs, because your logs only contain your traffic. It needs something outside the application making real requests to api.mistral.ai on a fixed interval and recording latency and error rate independently — otherwise the first hour of every incident is spent bisecting a prompt that was never the problem.

Frequently Asked Questions

Related Guides

• Mistral AI API Error Codes Explained
• Mistral AI Structured Output and JSON Mode
• Mistral AI API Pricing Guide
• Mistral AI API Latency Guide
• Mistral AI API Rate Limits Explained
• How to Test and Mock the Mistral AI API
• Mistral AI API Monitoring Guide
• Live Mistral AI Status Check