Mistral API 504 Gateway Timeout

A 504 is a statement about a clock, not about a failure. Some intermediary decided in advance how long it was willing to wait, the answer did not arrive inside that window, and the intermediary wrote an error of its own. Nothing in the status code tells you that Mistral failed. It tells you that a deadline expired, and deadlines are configuration — which means the fault is findable by reading a number rather than by guessing.

Whose clock is the whole question. It might be Mistral's edge. It might just as easily be your ingress controller, a service-mesh sidecar, a corporate egress proxy, a platform request limit, or the timeout argument you passed to your own SDK three releases ago and have not looked at since. These produce an identical status code and entirely different remediation.

504 against the rest of the 5xx family

These five get filed together as “the API is broken” and then handed one retry policy, which guarantees the policy is wrong for at least three of them. The last column is the one that matters, because it is the only one that changes what you do next.

502 and 504 are the two gateway-authored codes and they are routinely conflated. The difference is simple and it changes your diagnosis: a 502 means the gateway got an answer it could not use, so the evidence is in the response body; a 504 means it got nothing and stopped waiting, so the evidence is in the elapsed time. The unusable-reply case is covered in theMistral 502 guide, the capacity case in theMistral 503 guide, and throttling in theMistral 429 guide.

Why Mistral returns 504 specifically: several front doors, several deadlines

Mistral is unusual among the open-weight providers in how many independent ways there are to reach the same model. La Plateforme is one. Azure AI Foundry is another. Bedrock, Vertex and a handful of marketplace listings are others, and self-hosted deployments of the open-weight releases are a category of their own. These share a model name and share nothing else — different edges, different capacity pools, different request deadlines, different status pages.

That matters more for 504 than for any other status code, because 504 is definitionally about a deadline, and each of those front doors sets its own. A marketplace endpoint that enforces a sixty-second request limit will return 504 on a request that la Plateforme would have completed, and la Plateforme's status page will be green throughout because nothing about the incident touched it. Teams lose entire incidents to this, refreshing a status page for infrastructure they are not using.

The long-context case is the one place where the deadline is genuinely close to the work. Mistral's larger context windows invite document-sized prompts, and prompt processing time scales with input length in a way that is easy to underestimate — a request with a hundred thousand tokens of input spends real seconds before the first output token exists. If your 504s correlate with input size rather than with time of day, you are not looking at an incident at all; you are looking at a deadline that was set for short prompts and is now being handed long ones.

Sample rather than probe once, and record all three columns. One request cannot tell a total failure apart from the far more common partial one:

for i in $(seq 1 10); do
  curl -s -o /dev/null -w "%{http_code} %{time_total}s %{content_type}\n" \
    --max-time 90 \
    https://api.mistral.ai/v1/chat/completions \
    -H "Authorization: Bearer $MISTRAL_API_KEY" \
    -H "Content-Type: application/json" \
    -d '{"model":"mistral-large-latest","messages":[{"role":"user","content":"ping"}],"max_tokens":1}'
done

The status code says whether it failed, the elapsed time says which timer expired, and the content type says who wrote the error. Set --max-time deliberately higher than the deadline you are investigating, otherwise curl becomes the shortest timer in the path and you measure your own probe instead of the system.

The Mistral trap: log the resolved endpoint, because the model name is not evidence

A config line reading mistral-large-latest tells you nothing about where the request went. The same string is valid against la Plateforme, against a marketplace listing, and against a self-hosted deployment, and those fail independently. Emit the fully-resolved base URL on every error alongside the status code — one log line, and it removes the most expensive ambiguity in the whole failure mode.

Then check the deadline that endpoint enforces, which is frequently not documented next to the model. Cloud marketplace front doors tend to impose platform-wide request limits that have nothing to do with Mistral, and those limits are usually the shortest deadline in the path — meaning they fire first and their error is the one you see.

Finally, separate your 504 metrics by input length. If the failures live entirely in the long-prompt tail, the fix is chunking the input or moving to streaming, not raising a timeout — and certainly not failing over to another provider, which will encounter the same physics with a different logo.

The streaming case, where the 504 stops happening at all

Streaming is the most underrated fix for 504s, and it works for a reason that is easy to miss: most gateway timeouts are idle timeouts rather than total-duration limits. They fire when no bytes have moved for N seconds, not when the request has been open for N seconds. A streamed response puts bytes on the wire as soon as the first tokens exist, which resets the idle timer continuously and keeps a long generation alive under a deadline that would have killed the buffered equivalent.

• Check which kind of timeout you have. An idle timeout is defeated by streaming; a hard request-duration cap is not. They are configured separately and confusing them wastes a day.
• Assert on the terminal event. An iterator finishing is not the same as a stream completing. If the terminal marker never arrived, the response was truncated no matter what the status code said.
• Count partial responses as their own class. Not a success, not a 5xx. On a streamed path they are the metric that moves first, and a status-code-only dashboard stays flat through the entire incident.
• Decide the resume policy in advance. Re-issuing the whole prompt is the honest default and it bills twice; showing the partial output with a visible label is often the better product answer.

A retry policy that fits a 504 specifically

504 is the 5xx where the standard retry advice is most likely to hurt you, because the one thing a 504 does not tell you is whether the work happened. Four rules keep the retry from becoming its own incident:

• Assume the request may have succeeded. Your gateway stopped listening; the upstream did not necessarily stop working. Send an idempotency key for anything with a side effect, and expect a retried completion to be billable twice.
• Do not retry into the same deadline. If the timeout was too short for this request, a second identical attempt will expire identically. Change something — stream it, shrink it, or route it to a path with a longer deadline — or do not retry at all.
• Budget by ratio, not by count. Allow retries as a fraction of recent successes so the allowance collapses on its own when almost everything is failing, with no deploy required. See the Mistral retry budget guide.
• Retry against a deadline, not a counter. If the user-facing request gave up eight seconds ago, attempt three is load on behalf of nobody. Propagate the deadline and skip attempts that cannot finish inside it.

If 504s persist past a couple of attempts they have stopped being transient, and the failover machinery should take over — which only works if it was kept warm. TheMistral failover guidecovers keeping a second path tested rather than merely configured, and thecircuit breaker guidecovers stopping the retries automatically.

Frequently Asked Questions

Related Mistral Guides

• Mistral API 502 Bad Gateway
• Mistral API 503 Service Unavailable
• Mistral API Timeout Errors
• Mistral API 429 Too Many Requests
• Mistral API Latency Guide
• Mistral Streaming Errors Guide
• Mistral API Error Codes Reference
• Mistral API Monitoring Guide