Together AI API 429 Too Many Requests: Why It Fires and How to Clear It (2026)

A 429 is the only error in the 4xx range that is not a complaint about your request. The JSON was valid, the key was accepted, the model existed — and the service declined to run it because of how much you had already asked for. That makes it the one error where reading the payload gets you nowhere and reading the response headers gets you everything.

It is also the error most often confused with a Together AI outage. A 429 means the service is up and working precisely as designed; it is enforcing an allowance. If you are here because requests started failing, the first job is to establish which of the two you are looking at, because the response to a throttle and the response to an incident have almost nothing in common.

30-second triage: read the x-ratelimit-remaining-* headers on the failing response, not your dashboard. If remaining is at or near zero, this is your allowance and nothing about Together AI is broken. If the headers look healthy and calls still fail, check live Together AI status — you may be reading a 429 emitted by your own gateway rather than by api.together.xyz.

429 is not 503, and the fix is the opposite

These two get filed together as “capacity errors” and then treated with the same retry policy, which is how teams end up applying a fix that cannot work. The distinction is about whose consumption caused it, and it determines everything downstream.

	429 Too Many Requests	503 Service Unavailable
Cause	Your account exceeded a written allowance	The fleet had no capacity for anyone
Does slowing down help?	Yes — immediately and reliably	Often not at all
Does paying more help?	Yes — limits are an account property	No — billing does not create hardware
Failing over to another provider	Works, but you are exporting your own pacing bug	The correct response
Shows on a status page?	Never — the service is healthy	Usually, if it is widespread

The row that costs the most money is the fourth one. Failing over to a second provider on 429 feels like resilience and is frequently the opposite: you have taken a load pattern that one provider already told you was too aggressive and pointed it at a provider that has not told you yet. Failover belongs on 5xx. A 429 belongs in a queue. The full treatment of the 5xx side is in the Together AI 503 guide.

📡

Recommended

Tell a Throttle From an Outage in Seconds

External checks against your AI endpoints run from outside your infrastructure, so you can see immediately whether the failure is your allowance or the provider's fleet — instead of debating it while requests fail.

Try Better Stack Free →

Read the headers, not the dashboard

A rate-limit dashboard aggregates. A 429 does not. The response that refused you carries headers scoped to the exact allowance, model and window that produced the refusal, and that is the only evidence that is actually about your error. Together AI returns x-ratelimit-limit, x-ratelimit-remaining and x-ratelimit-reset, scoped to the model in the request, and the gap between what those say and what a dashboard chart implies is where most of the confusion in this category lives.

curl -i -s -D /dev/stderr -o /dev/null \
  https://api.together.xyz/v1/chat/completions \
  -H "Authorization: Bearer $TOGETHER_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model":"meta-llama/Llama-3.3-70B-Instruct-Turbo","messages":[{"role":"user","content":"ping"}],"max_tokens":1}' \
  2>&1 | grep -i "ratelimit\|retry-after\|^HTTP"

Capture those headers on every response, not only on failures. The value of the remaining counters is that they let you see the allowance being consumed before it is exhausted; harvesting them only from 429s means you have instrumented the moment it is already too late to act. A gauge of remaining-headroom over time turns this entire class of incident into something you watch approaching rather than something that arrives.

The Together AI trap: the limit belongs to the model, and your model string is a moving target

On a single-model provider you can reason about “your rate limit” as one number. On a marketplace that phrase has no referent. Each serverless model carries its own requests-per-minute and tokens-per-minute allowance, set by how expensive that model is to serve, and the spread between a small chat model and a large reasoning model is wide enough that code which was comfortably inside the limit becomes a throttled mess purely by changing a string in a config file.

That makes the fallback chain the most common cause of a surprising Together 429. A router that spills from a primary model to a second one on error is, by construction, spilling onto a different allowance — and if the fallback is the larger model, you have arranged for every capacity problem to be met with a stricter limit. Fallbacks should be chosen by the headroom of the target, not merely by capability, and the allowance for each model in the chain should be recorded next to the model id rather than discovered during an incident.

The structural exit from serverless 429s is a dedicated endpoint, and it is worth understanding what changes. A dedicated deployment is bounded by concurrency rather than a published per-minute allowance, so requests beyond its capacity queue and eventually time out instead of returning 429. That is usually an improvement, but it is a different failure to monitor: your 429 rate goes to zero and your p99 latency becomes the thing that tells you the endpoint is saturated.

Fix the pacing before you fix the retries

Retrying a 429 harder is the reflex and it is the wrong order of operations. A retry loop against a throttle converts one refused request into several refused requests, which consumes the very allowance you are waiting to recover. The sequence that actually works starts upstream of the retry, and each step removes load rather than rescheduling it:

Cap concurrency at the client. A single semaphore sized to your published allowance is worth more than any backoff policy, because it prevents the 429 rather than reacting to it. Most teams discover their real concurrency is set by whatever their worker pool happened to default to.
Queue instead of failing. Work nobody is watching — enrichment, backfills, scheduled summaries — should accept a delay rather than an error. Draining a queue at a fixed rate makes your consumption a constant instead of a spike.
Honour Retry-After exactly. When it is present it is the only number in the exchange that reflects what the service knows. Waiting less is not clever; it guarantees the next attempt is refused too.
Back off with full jitter. Randomise across the whole window rather than adding a wobble. Deterministic backoff synchronises your own fleet into waves that recreate the burst that caused the throttle.
Separate interactive from batch traffic. Different keys, different queues, different priorities. Otherwise a nightly job decides your users’ error rate, and it will.

Once pacing is in place, the retry policy has far less work to do. Size it as a budget expressed as a ratio of retries to successes, so it collapses on its own when almost everything is failing — the detail covered in the Together AI retry budget guide.

When raising the limit is the right answer

Pacing has a floor. If your steady-state demand genuinely exceeds the allowance, queueing only converts an error rate into a latency figure and eventually a backlog that never drains. Three signals say the constraint is the quota rather than your behaviour: the remaining-headroom gauge sits near zero across the whole day rather than at peaks; your concurrency cap is already at the published limit; and the queue depth trends upward over a week instead of oscillating.

When those hold, raise the ceiling rather than tuning further. What to ask for and how to evidence it is in the Together AI quota increase guide, and the published baselines you are arguing against are in the Together AI rate limits reference. Bring the header data with you — a request backed by a remaining-headroom timeseries is a different conversation from one backed by an assertion.

Frequently Asked Questions

What does a 429 from the Together AI API actually mean?

It means the request itself was fine and the service refused it because of how much you had already asked for inside a measured window. Nothing in the payload needs to change for the identical call to succeed a moment later, which puts it in a different category from every other 4xx: a 400 or a 401 will keep failing until you edit something, and a 429 will keep succeeding once you slow down. Read it as a scheduling constraint, not a defect. The corollary matters too — a 429 is proof the service is healthy and enforcing policy, so it should never be counted as Together AI downtime in your own availability numbers.

Is a 429 the same as Together AI being down?

No, and treating them the same produces the wrong response in both directions. A 429 says the service is up and deliberately declining your excess; a 5xx says the service could not serve the request at all. The clean test is what happens when you reduce load: cut concurrency and 429s stop almost immediately, while a genuine outage carries on regardless of how politely you ask. Split the two in your metrics and your alerting, because they need opposite responses — 429s want a queue and slower pacing, outages want failover to a second provider. A dashboard that lumps all failures into one error-rate line makes this distinction impossible to see at exactly the moment it matters.

Why does one Together AI model 429 while another works fine on the same key?

Because they are separate allowances. Together AI meters serverless inference per model, not per account, so the requests-per-minute and tokens-per-minute figures attached to a large reasoning model are much lower than those on a small chat model, and your key holds both simultaneously. This is why a fallback chain can make throttling worse rather than better: spilling from a model you exhausted onto a larger, stricter one meets your capacity problem with a tighter limit. Keep a table of model id to published allowance in the same config that holds the fallback order, choose fallbacks by headroom rather than by capability alone, and treat a dedicated endpoint as the answer when a single model is persistently the constraint.

How should I retry a 429 without making it worse?

With a policy rather than a loop, and only after you have capped concurrency. Honour a Retry-After header exactly when one is present, and otherwise back off exponentially with full jitter so a fleet of clients does not resynchronise and re-burst the instant the window resets. Cap attempts against a deadline rather than a fixed count, so a request nobody is waiting for is abandoned instead of retried. Express the allowance as a retry budget — a ratio of retries to successes — which collapses automatically when most calls are failing, the case where naive retries do the most damage. And never retry a 429 in a tight loop with a fixed short delay: that is not a retry policy, it is a second source of the load that caused the problem.

Will upgrading my Together AI plan stop the 429s?

Usually yes, because unlike a 503 a 429 is genuinely an account property — the allowance is written down and attached to you. That makes upgrading a legitimate fix rather than a superstition. It is the right fix when your remaining-headroom gauge sits near zero across the whole day rather than at peaks, your client concurrency is already capped at the published limit, and your queue depth trends upward across a week. It is the wrong fix when the 429s are bursty and cluster around a specific job, because a higher ceiling simply raises the level at which the same unpaced burst trips it, and you will be back with a bigger bill and the same error.

Should I fail over to another provider when I get a 429?

Rarely, and it is the most expensive habit in this category. A 429 means one provider has already told you your load pattern is too aggressive for the allowance you hold; routing that same pattern to a second provider exports the problem rather than solving it, and you will trip the new provider's limits too, only later and with a warm cache of false confidence in between. Failover is the correct response to 5xx, where the fault genuinely is on the provider side. For a 429 the correct responses are a client-side concurrency cap, a queue for work nobody is watching, and a quota increase if steady-state demand truly exceeds the ceiling. The one legitimate exception is a deliberate multi-provider architecture where you are load-balancing across allowances by design rather than reacting to an error.

Related Together AI Guides

Stop Guessing Whether It Is You or Together AI

API Status Check watches Together AI and the rest of your stack from outside your infrastructure, so a throttle never gets debugged as an outage — and a real outage never gets dismissed as a throttle.

Start Your Free Trial →

Together AI API 429 Too Many Requests

Live Together AI status right now