Cohere API 503 Service Unavailable: What It Means and How to Respond (2026)

Most API errors are an argument about the request. A 503 is not. The JSON was valid, the key was accepted, the model existed — and the service still said no, because at that instant it had nowhere to run the work. Reading it as a bug sends teams into a debugging session that cannot possibly find anything, because there is nothing in the code to find.

30-second triage: run the same call with curl from outside your deployment. 503 there too? It is Cohere — check live Cohere status and skip to the retry policy below. Succeeds in curl? The 503 is being generated by something between your app and api.cohere.com, and the fix is on your side.

Which 5xx you actually received

The status code narrows the cause far more than the message body does, and the four you will see from an AI endpoint mean genuinely different things. Getting this wrong is expensive: three of the four are worth retrying and one usually is not.

Code	What it means	Retry?
`429`	Your key exceeded its own allowance	Yes, after backoff — but fix your pacing
`500`	Something broke while handling the request	Once or twice; a persistent 500 is a bug report
`502`	A gateway got a bad response from upstream	Yes — usually brief and self-clearing
`503`	No capacity available for the request	Yes, with jitter and a budget
`504`	Upstream took longer than the gateway allowed	Only if the call is idempotent

One caveat that catches everyone: a 503 in your logs did not necessarily come from api.cohere.com. Load balancers, service meshes and your own web framework all emit 503 when they are shedding load, and the code is identical. Log the response headers alongside the status — the provider's responses carry identifying headers that your own gateway's do not.

📡

Recommended

Know Which Side Returned the 503

External checks against your AI endpoints run from outside your infrastructure, so you can tell a provider capacity refusal from your own gateway shedding load — in seconds, not after twenty minutes of arguing.

Try Better Stack Free →

Why Cohere returns 503 specifically

Cohere serves three quite different workloads behind one key: chat, embed and rerank. They do not share a failure mode. A 503 on embed in the middle of a bulk indexing run is usually a statement about the load you are generating, while a 503 on chat during normal traffic is a statement about the service. Treating them as one number is why Cohere incidents get misdiagnosed.

Because chat, embed and rerank have separate characteristics, a liveness check against /v2/chat does not tell you embed is healthy. Probe each endpoint your product actually uses, and label the metrics by endpoint — a single ’Cohere error rate’ panel will average a broken indexing pipeline into a healthy chat path and show you nothing.

curl -i -s -o /dev/null -w "%{http_code} %{time_total}s\n" \
  https://api.cohere.com/v2/chat \
  -H "Authorization: Bearer $CO_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model":"command-r-plus","messages":[{"role":"user","content":"ping"}],"max_tokens":1}'

Run that on a short loop during an incident rather than once. A single sample cannot distinguish a total outage from partial capacity, and partial is the far more common state: ten samples returning seven 200s and three 503s is a completely different operational situation from ten out of ten failing, and it calls for a different response.

The Cohere trap: a 503 mid-index is not the same error as a 503 mid-chat

Bulk embedding jobs are the single most common source of Cohere 5xx in production, and they are self-inflicted: a job that fires thousands of documents as fast as the client allows will find the ceiling, and the naive fix — retry the failed batch immediately — reapplies exactly the load that caused it. Worse, a partially-failed index is not merely incomplete, it is inconsistent, and a retrieval system querying a half-built index returns confident nonsense with no error at all. Checkpoint the job by document id and resume, never restart.

A retry policy that helps instead of hurting

The default reflex — three immediate retries — is the worst possible response to a capacity error. It triples your offered load at the exact moment the service has none to give, and every other client on the platform is doing the same thing at the same time. Four rules turn retries from an amplifier into a recovery mechanism:

Honour Retry-After. When the response carries one, it is the only number in the exchange that reflects what the service actually knows. Waiting less is not clever.
Back off with full jitter. Randomise the delay across the whole window rather than adding a small wobble. Deterministic backoff makes every client wake at the same instant and hit the first sliver of recovered capacity together.
Budget by ratio, not by count. Allow retries as a fraction of recent successes. When almost everything is failing the allowance collapses on its own, without a deploy.
Retry against a deadline. If the user-facing request gave up ten seconds ago, attempt three is pure load on behalf of nobody. Propagate the deadline and skip any attempt that cannot finish inside it.

The full treatment, including how to size the budget and ramp concurrency back after recovery, is in the Cohere retry budget guide.

What your product does while it lasts

Retries buy seconds. A capacity event that runs for twenty minutes needs a product answer, and there are only three honest ones. Decide which applies to each call site before an incident, because the decision is a product decision and nobody makes good ones at 2am.

Fail over

Route to a second provider behind the same interface. Best for calls where any competent model will do. Needs the fallback kept warm and tested, not merely configured. See the Cohere failover guide.

Degrade visibly

Serve a cached or simpler result and say so. Far better than a spinner: users forgive a labelled downgrade and do not forgive silence.

Queue and defer

For work nobody is watching, accept it, persist it, and drain when capacity returns. Turns an outage into latency instead of an error.

What none of the three tolerate is finding out late. The gap between the first 503 and the moment someone notices is where the damage happens, and it is the one part of this that is entirely within your control — external monitoring closes it to seconds regardless of which of the three you chose.

Frequently Asked Questions

What does a 503 from the Cohere API actually mean?

It means the request was well-formed, your key was accepted, and the service declined to run it because it had no capacity to spare at that moment. Nothing about the request needs to change for it to succeed later — which is precisely what separates a 503 from a 400 or a 401, where retrying the identical payload can only produce the identical error. Read it as a scheduling outcome rather than a defect: the same JSON that failed at 14:02 will very often succeed at 14:03, and the engineering question is not what is wrong with the call but what your system does during the minute in between.

Is a 503 the same as a 429 rate limit?

No, and conflating them sends you to the wrong fix. A 429 is about you: your key exceeded an allowance that is written down somewhere, and the remedy is your own pacing, a higher tier, or a queue. A 503 is about the service: capacity was not available for anyone in that position, and your own consumption may have been entirely modest. The tell is what happens when you slow down. Reduce concurrency and 429s disappear almost immediately; 503s often continue at the same rate, because you were never the cause.

Should I retry a 503, and how?

Yes, but with a policy rather than a loop. Honour a Retry-After header when one is present, and otherwise back off exponentially with full jitter so that a fleet of clients does not synchronise and hit the service in waves. Cap total attempts against a deadline rather than a fixed count, so a request nobody is waiting for any more is abandoned instead of retried. And keep a retry budget expressed as a ratio of retries to successes, which collapses on its own when most calls are failing — the case where naive retries do the most damage.

What should I never retry after a 503?

Anything that may have already had an effect. A 503 usually means the work never started, but a 503 returned from an intermediary after the request was accepted is indistinguishable from one returned before, and a retried batch submission or long-running job can quietly become two. Any call that creates durable server-side state needs an idempotency key or a ledger you check before re-sending. For plain completions the risk is cost rather than correctness, but a retried streaming call that had already emitted tokens still bills for them.

How do I tell a Cohere 503 apart from a failure in my own infrastructure?

Take your application out of the path. Run one curl against the exact endpoint your code calls, from a machine outside your deployment, with a valid key. A 503 there is the provider. A success there while your app still fails means the failure is between your app and api.cohere.com — your proxy, your egress, your DNS, or a gateway of your own returning its own 503, which is a genuinely common confusion because the status code is identical and the body is not.

My Cohere embedding job returns 503 halfway through. Do I restart it?

No — restarting is what turns a delay into a data problem. Re-embedding documents you already stored wastes tokens, and if the job restarts under a different model version than it began with, the vectors in your index are no longer comparable and every similarity score in that neighbourhood is quietly wrong. Checkpoint instead: record the last successfully embedded document id, resume from it, and pin the model identifier explicitly for the whole run rather than accepting a default. Then halve the batch size and add real backoff on the way back up, because a 503 mid-job is nearly always your own concurrency finding the ceiling rather than the service being down.

Related Cohere Guides

Find Out About the 503 Before Your Users Do

API Status Check watches Cohere and the rest of your stack from outside your infrastructure and alerts you the moment capacity errors start — so failover happens on your schedule, not after the first support ticket.

Start Your Free Trial →

Cohere API 503 Service Unavailable