Groq API 502 Bad Gateway: Who Actually Returned It and How to Clear It (2026)

A 502 is the only 5xx that is definitionally written by someone other than the component that failed. A gateway asked an upstream for a response, got something it could not use — an empty body, a reset connection, a malformed reply — and substituted an error of its own. Everything useful about a 502 follows from that one fact: your request was fine, the model may never have seen it, and the error you are reading was authored by a middleman.

Which middleman is the entire question. It might be Groq's own edge. It might equally be your ingress controller, a service-mesh sidecar, a corporate egress proxy, a CDN, or a routing library sitting between your code and the provider. These produce an identical status code and wildly different remediation, and teams routinely spend an incident refreshing a status page for a failure their own infrastructure emitted.

30-second triage: look at the response Content-Type. JSON with a structured error envelope means Groq answered. HTML with a <title>502 Bad Gateway</title> and a server banner means an intermediary answered and the model tier never saw your call. If it is JSON and sustained, check live Groq status and fail over. If it is HTML, read your own gateway logs.

502 against the rest of the 5xx family

These five get filed together as “the API is broken” and then handed one retry policy, which guarantees that the policy is wrong for at least three of them. The column that matters is the last one, because it is the only one that changes what you do next.

Code	Who authored it	Correct response
`429`	Groq, deliberately, about your allowance	Slow down. Never fail over — you export your own pacing bug
`500`	The service, about an unhandled fault	Retry once or twice; a persistent 500 is a bug report
`502`	An intermediary — possibly yours	Identify the hop first, then one jittered retry
`503`	The service, about having no capacity	Retry with a budget, then fail over or degrade
`504`	An intermediary, about a deadline it set	Retry only if idempotent; consider raising the timeout

502 and 504 are siblings and both are written by gateways — the difference is that a 504 means the intermediary waited and gave up, while a 502 means it got an answer it could not use. Neither tells you the upstream failed. The capacity case is covered separately in the Groq 503 guide, and the throttling case in the Groq 429 guide.

📡

Recommended

Know Which Side Wrote the 502

External checks run from outside your own network, so a gateway failure inside your infrastructure looks different from a provider edge failure — instead of identical, which is what your application logs show you.

Try Better Stack Free →

Why Groq returns 502 specifically

Groq serves an OpenAI-compatible surface, and that compatibility is exactly what makes its 502s hard to attribute. The same base URL shape is accepted by LiteLLM, OpenRouter, a self-hosted gateway and half a dozen SDK wrappers, so a large share of production traffic that people describe as "calling Groq" is in fact calling something that calls Groq. When one of those intermediaries fails, it returns 502 in its own voice and the word Groq never appears in the response.

The timing signal is unusually clean here because Groq is unusually fast. Time-to-first-token on the LPU fleet is measured in tens of milliseconds, so a 502 that arrives after four or five seconds of wall-clock time did not come from a Groq generation that struggled — it came from an intermediary that waited, gave up, and synthesised a gateway error. Record total time alongside the status code and the attribution mostly makes itself.

Sample rather than probe once. One request cannot distinguish a total failure from the far more common partial one, and the two call for different responses:

for i in $(seq 1 10); do
  curl -s -o /tmp/body -w "%{http_code} %{time_total}s %{content_type}\n" \
    https://api.groq.com/openai/v1/chat/completions \
    -H "Authorization: Bearer $GROQ_API_KEY" \
    -H "Content-Type: application/json" \
    -d '{"model":"llama-3.3-70b-versatile","messages":[{"role":"user","content":"ping"}],"max_tokens":1}'
done

Three columns, three answers. The status code tells you whether it failed, the elapsed time tells you whether something waited before failing, and the content type tells you who wrote the error. Ten out of ten failing with a JSON body is an incident. Two out of ten failing with an HTML body is your own infrastructure having a moment, and no amount of provider failover will fix it.

The Groq trap: log the resolved base URL, not the model name

Because the request shape is OpenAI-compatible, the model string is not evidence of where the request went. A config that reads model: "llama-3.3-70b-versatile" tells you nothing about whether the bytes left your network for api.groq.com or for a proxy in the next availability zone. Teams routinely spend an incident reading the Groq status page while the 502 is being emitted by their own LiteLLM pod restarting under memory pressure.

The fix is one log line: emit the fully-resolved base URL on every error, next to the status code and the response content type. If the base URL is not api.groq.com, the Groq status page is not the document you need. If it is, and the body is HTML rather than JSON, you have learned that a CDN or load balancer in front of the fleet answered rather than the fleet itself — which is still not a model failure and still self-clears faster than a capacity event.

The streaming case, where the 502 never reaches your status-code metrics

If you stream, a large share of your gateway failures will never be counted as 502s at all. The response headers arrived long ago with a 200; the failure happens partway through the token stream, and what your client receives is a truncated event or an HTML error fragment spliced into a channel that was expecting server-sent events. Most SDKs raise that as a decoding error, so it lands in your logs as a parse failure in the JSON layer and your Groq error-rate panel stays flat through the entire incident.

Assert on the terminal event. An iterator finishing is not the same as a stream completing. If the terminal marker never arrived, the response was truncated, whatever the status code said.
Count partial responses as their own class. Not a success, not a 5xx. They behave differently and they are the metric that moves first.
Record bytes and tokens received on failure. A stream that died at token three is a different problem from one that died at token three hundred, and only one of them is worth showing the user.
Decide the resume policy in advance. Re-issuing the whole prompt is the honest default and it bills twice; showing the partial output with a visible label is often the better product answer.

A retry policy that fits a 502 specifically

502s are the most retry-friendly 5xx there is — they are usually brief, usually one hop, and usually gone by the time you look. That makes blanket “never retry 5xx” advice actively wrong here. Four rules keep the retry from becoming its own incident:

One quick jittered retry first. Randomise across the full window rather than adding a small wobble, so a fleet of clients does not resynchronise and re-burst against the first recovered backend.
Assume the work may have happened. The gateway could not read the response; that does not mean the upstream never produced one. Send an idempotency key for anything with a side effect, and expect that a retried completion can bill twice.
Budget by ratio, not by count. Allow retries as a fraction of recent successes so the allowance collapses on its own when almost everything is failing — no deploy required. See the Groq retry budget guide.
Retry against a deadline. If the user-facing request gave up eight seconds ago, attempt three is load on behalf of nobody. Propagate the deadline and skip attempts that cannot finish inside it.

If 502s persist past a handful of retries, they have stopped being transient and the failover machinery should take over — which only works if it was kept warm. The Groq failover guide covers keeping a second path tested rather than merely configured, and the circuit breaker guide covers stopping the retries automatically.

Frequently Asked Questions

What does a 502 from the Groq API actually mean?

It means some intermediary between your process and the model received an invalid, empty or truncated response from whatever it forwarded your request to, and wrote its own error rather than passing anything through. The crucial word is intermediary: a 502 is never authored by the component that failed, which is what makes it the hardest 5xx to attribute. Your request was almost certainly fine — 502 says nothing about your payload, your key or your model choice — so editing the request is wasted effort. The productive question is not what is wrong with the call but which hop in the path wrote the error, and that is answerable in about thirty seconds from the response itself.

How do I tell a Groq 502 from a 502 emitted by my own proxy?

Read the content type and the body, not the status code. Groq returns structured JSON error envelopes with recognisable fields and its own identifying response headers. Load balancers, ingress controllers, CDNs and corporate proxies return an HTML page — a short document with a title like "502 Bad Gateway" and a server banner at the bottom. If the body is HTML, the model tier never saw your request and the Groq status page is not the document you need. That single check resolves the majority of misattributed incidents, and it costs nothing to log the first 200 bytes of every error body alongside the status code so the evidence is already captured next time.

Is a 502 the same as Groq being down?

Not usually, and the difference matters because it changes what you should do. A genuine outage is broad, sustained and consistent across endpoints and keys. A 502 is characteristically brief, patchy and self-clearing — one hop lost one upstream connection for a moment. The test takes seconds: sample the endpoint ten times in a loop rather than once. Ten out of ten failing over several minutes is an incident and warrants failover; three out of ten failing is transient turbulence that a correct retry policy absorbs without anyone noticing. Reacting to a single 502 as though it were an outage is how teams cut over to a fallback provider for an event that had already resolved before the deploy finished.

Is it safe to retry a Groq 502?

Safer than most 5xx, with one real caveat. A 502 is typically transient and a single retry after a short jittered delay clears it, which is why blanket "never retry 5xx" advice is wrong here. The caveat is that a 502 tells you the intermediary did not get a usable response — it does not tell you the upstream never did the work. For a chat completion that is a billing question rather than a correctness one, and it is worth knowing that a retried request can be charged twice. For anything with a side effect, propagate an idempotency key so the retry is provably the same operation rather than a second one. Cap retries against a deadline rather than a fixed count, and use full jitter so a fleet of clients does not resynchronise on recovery.

Why does a 502 show up as a JSON parse error instead of a status code in my logs?

Because of streaming. On a streamed response your client has already received a 200 and started consuming server-sent events; if a hop fails mid-stream, what arrives is either a truncated event or an HTML error fragment spliced into a stream that was expecting JSON. Most SDKs surface that as a decoding exception, so the incident lands in your logs as a parse failure and never appears in the status-code panel at all. Handle stream termination explicitly — check for the terminal event rather than assuming the iterator ending means completion — and record partial-response events as their own failure class. Otherwise your dashboards will show a healthy Groq error rate throughout an incident your users are watching happen.

Groq is supposed to be fast — why would a 502 ever take seconds to arrive?

Because the seconds were not spent generating. On the LPU fleet, first tokens land in tens of milliseconds and a full short completion is typically under a second, so a five-second failure is a gateway sitting in its own read timeout waiting for an upstream that never answered, then writing a 502 when the timer expired. That upstream might be Groq, but far more often it is a proxy layer between your code and Groq — an egress proxy, a service mesh sidecar, or a routing library like LiteLLM. The practical rule: if elapsed time on a 502 is an order of magnitude above your normal Groq latency, stop reading the Groq status page and start reading your own gateway logs, because the number is telling you something waited.

Should I fail over to another provider on a 502?

Only after you know which hop wrote it, which is the whole point of the content-type check. If the 502 came from an intermediary you own — an egress proxy, a mesh sidecar, an ingress controller — failing over to a second provider routes the same traffic through the same broken hop and changes nothing except your bill and your confidence. If it genuinely came from Groq's edge and is sustained across a sampled loop, failover is the correct response and should be a config flip you have already tested rather than a change you write during the incident. The middle case — brief and patchy — wants neither: it wants one jittered retry and no human involvement at all.

Related Groq Guides

Catch the 502 Before Your Users Report It

API Status Check watches Groq and the rest of your stack from outside your own network, so you find out whether the gateway that failed was theirs or yours — in seconds, not after twenty minutes of reading a status page that was never going to mention it.

Start Your Free Trial →

Groq API 502 Bad Gateway

Live Groq status right now