Groq API Timeout Error: Why Requests Hang and How to Fix Them (2026)

Groq is the fastest inference endpoint most teams have ever pointed at, which is exactly why a timeout there is so confusing. If the model normally answers in under a second, a request that runs for thirty is almost never the model.

A timeout is not a response. Nothing came back, so nothing tells you whether the request reached Groq, whether it was processed, or whether something in between gave up first. Every other error hands you a status code to search for; this one hands you silence. That is why teams lose hours to it — the debugging instinct is to check the provider's status page, which is the layer least likely to be responsible.

60-second triage: run the same call from your terminal with a deliberately long --max-time. If curl returns a completion, Groq is healthy and something in your stack killed the request — keep reading. If curl hangs or returns 5xx as well, check live Groq status and the error code reference instead.

Four layers can kill the request, and only one is Groq

Before changing any value, find out who enforced the limit. The shortest timeout in the chain wins, and it is almost never the one you configured — it is the one you inherited from a framework default or a hosting platform.

Layer	Typical limit	How it looks
Your HTTP client / SDK	Often a default you never set	An abort or `ETIMEDOUT` raised inside your own code
Serverless function or gateway	10s, 29s, 30s, 60s — round numbers	A platform-generated 504, and your handler's logs stop mid-execution
Proxy, CDN or load balancer	Idle timeout, resets on bytes	Non-streaming calls die, streaming calls survive
Groq itself	Genuine slowness or an incident	curl fails too, from every machine, at the same time

The round numbers in that second row are the most useful clue on this page. A request that dies at almost exactly 10, 29, 30 or 60 seconds was killed by infrastructure, not by a model. If Groq is genuinely degraded you will usually see it as elevated 503s and queue errors rather than silence, so a pure hang points harder at your own stack than it would with a slower provider.

📡

Recommended

Know Which Layer Failed Before You Start Guessing

External checks running against your AI endpoints answer the only question that matters mid-incident: is the provider failing, or is it my deployment?

Try Better Stack Free →

Prove it with curl before you change code

Take your application entirely out of the picture. This runs the same request with a long ceiling and prints a full timing breakdown, so you can see where the seconds actually went:

curl -sS -o /dev/null --max-time 120 \
  -w "dns=%{time_namelookup}s connect=%{time_connect}s tls=%{time_appconnect}s firstbyte=%{time_starttransfer}s total=%{time_total}s\n" \
  https://api.groq.com/openai/v1/chat/completions \
  -H "Authorization: Bearer $GROQ_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model":"llama-3.3-70b-versatile","messages":[{"role":"user","content":"ping"}]}'

Read it from the left. A large connect or tls figure is a network or handshake problem and has nothing to do with the model. A small connect with a large firstbyte means the connection was established instantly and Groq spent the time thinking — that is real processing latency, and the answer is a longer budget or a smaller request, not a retry. If the whole thing completes well inside your application's configured timeout, the request never had a chance and your own limit is the bug.

Why a Groq timeout is almost never slow inference

Groq runs on custom LPU hardware and routinely returns hundreds of tokens per second. When a call to llama-3.3-70b-versatile takes a second or two, that is normal. When it takes thirty seconds and then dies, something structural happened before or around the generation — not during it.

The most common cause is queueing under capacity pressure. Groq's free and low-tier lanes are heavily subscribed, and when the pool is saturated your request waits for a slot rather than failing fast. From your side that wait is indistinguishable from a slow model: the socket is open, no bytes have arrived, and your client's timer is running. The queue wait counts against your timeout even though no tokens have been generated yet.

The second cause is your own platform. A Groq call that completes in 1.2 seconds locally will still be killed at 10 seconds on a Vercel Hobby function if your handler is doing other work first, and a cold Lambda that spends 4 seconds initialising has only the remainder of its budget left. Because Groq is fast, teams set aggressively short client timeouts — 5 seconds is common — and those tripwires fire the first time the provider has a bad minute.

Set two timeouts, not one

A single overall deadline is the wrong shape for this problem. Connecting should be fast and failing to connect should fail immediately; generating can legitimately be slow. Collapsing both into one number means either network failures hang for the full budget, or slow-but-healthy requests get killed. Split them:

// Short connect budget, generous read budget.
async function callGroq(body: unknown) {
  const ac = new AbortController();
  const connectGuard = setTimeout(() => ac.abort(new Error('connect-timeout')), 5_000);
  const readGuard = setTimeout(() => ac.abort(new Error('read-timeout')), 30_000);

  try {
    const res = await fetch('https://api.groq.com/openai/v1/chat/completions', {
      method: 'POST',
      signal: ac.signal,
      headers: {
        Authorization: `Bearer ${process.env.GROQ_API_KEY}`,
        'Content-Type': 'application/json',
      },
      body: JSON.stringify(body),
    });
    clearTimeout(connectGuard); // headers arrived — connection is alive
    if (!res.ok) throw new Error(`upstream ${res.status}`);
    return await res.json();
  } finally {
    clearTimeout(connectGuard);
    clearTimeout(readGuard);
  }
}

Then make sure the platform agrees with you. A 60-second client timeout inside a function capped at 10 seconds is decoration — the platform kills the invocation first and your carefully chosen value never applies. Raise the function's duration limit to sit above the client timeout, or move the call out of the request path entirely.

Streaming turns a total failure into a partial one

The single highest-leverage change is to stop waiting for the whole response. With stream: true the first bytes arrive quickly, which keeps every idle timer in the chain — proxies, load balancers, serverless runtimes — from concluding the connection is dead. Those idle timers are what kill most non-streaming calls long before the total duration cap is reached.

It also changes what failure costs. A buffered request that dies at the deadline returns nothing at all, and you pay for tokens the user never saw. A stream that breaks after several hundred tokens has already delivered something useful and gives you a point to resume from.

With streaming, replace the total deadline with an idle deadline that resets on every chunk. A request that has produced tokens two seconds ago is healthy no matter how long it has been running; one that has produced nothing for thirty seconds is stuck regardless of how recently it started:

// Idle timeout — resets whenever a chunk arrives.
const reader = res.body!.getReader();
let idle = setTimeout(() => reader.cancel('stalled'), 30_000);

while (true) {
  const { done, value } = await reader.read();
  if (done) break;
  clearTimeout(idle);
  idle = setTimeout(() => reader.cancel('stalled'), 30_000);
  yield value;
}
clearTimeout(idle);

Retries that help, and retries that make it worse

Timeouts are the most dangerous class of error to retry blindly, because you do not know what happened. The request may have failed instantly, or it may have been fully processed with the response lost on the way back — and retrying the second case pays for the work twice while adding load to a system that was already struggling.

Retry on zero bytes received. Nothing was generated, so nothing is duplicated. This is the safe case.
Do not blindly retry a mid-stream failure. Tokens were produced and billed; resume or degrade instead.
Always use jitter. Fixed backoff synchronises every client into waves that arrive together and re-fail together.
Cap attempts at two or three, then fail loudly. Unbounded retries turn a provider blip into a self-inflicted outage.
Put a circuit breaker in front. When failures cross a threshold, stop calling for a cooling period — see the Groq failover guide.

And measure the right number. Averages hide timeouts completely: a p50 of 900 milliseconds tells you nothing about the 1% of requests sitting at your ceiling. Alert on p99 and on the timeout rate itself, because that tail is the entire population of users experiencing the bug.

Frequently Asked Questions

Why does the Groq API time out when it is supposed to be fast?

Because the timer starts when you open the connection, not when generation starts. Groq's speed applies to token generation; it does not shorten the queue wait in front of it. Under capacity pressure on the free and lower paid tiers, a request can sit waiting for an inference slot for tens of seconds before the first token appears, and your client counts every one of those seconds. A second, less obvious cause is that the platform running your code has its own ceiling — a Vercel Hobby function stops at 10 seconds and an API Gateway route at 29 — so the request is killed at your edge while Groq is still working on it. Log the elapsed time on the server side and compare it against your platform limit before assuming Groq is at fault.

How do I tell a Groq timeout apart from a Groq outage?

Run the same request from a terminal with curl and a generous timeout. If curl returns a completion in a few seconds, Groq is healthy and the timeout belongs to your application or hosting platform. If curl also hangs, or returns 5xx, check live Groq status and the incident feed. The distinguishing signal is consistency: an outage fails every request from every client and location at once, while a timeout in your stack tends to be intermittent, concentrated in one deployment or region, and often correlates suspiciously well with a round number like 10, 29 or 30 seconds — the fingerprint of a platform limit rather than a provider fault.

What timeout should I set on Groq API calls?

Set the read timeout to roughly three times the p99 latency you actually observe, not to the average, and never leave it at the client library default. For typical chat completions on Groq that lands around 20 to 30 seconds, which is long enough to absorb a queue wait but short enough that a genuinely stuck request does not hold a connection open. Set a separate and much shorter connect timeout — 3 to 5 seconds — so a network-level failure fails immediately instead of consuming the whole read budget. If you are streaming, do not use a single overall deadline at all; use an idle timeout that resets on every chunk.

Does streaming prevent Groq timeouts?

It prevents most of the ones that matter. With stream: true the first token typically arrives in a few hundred milliseconds, which keeps proxies, load balancers and serverless platforms from concluding the connection is dead — many of them kill a socket that has produced no bytes long before the total duration limit is reached. It also converts a hard failure into a partial one: a stream that breaks after 400 tokens gives the user something and gives you a resumption point, where a buffered request that times out at 30 seconds returns nothing at all. Streaming does not raise your platform's total duration cap, so a very long generation on a short-limit function can still be cut off mid-stream.

Should I retry a Groq request that timed out?

Only with an idle-timeout distinction and a cap. A request that timed out with zero bytes received is safe to retry — nothing was generated and you are almost certainly retrying a queue wait. A request that timed out mid-stream has already consumed tokens you were billed for, and a blind retry doubles that cost while adding load to the pool that was already saturated. Use exponential backoff with jitter, cap it at two or three attempts, and put a circuit breaker in front so a sustained incident does not turn into a retry storm that keeps your own latency pinned while Groq recovers.

Related Groq Guides

Stop Guessing Whether It Is Groq or You

API Status Check monitors Groq and the rest of your stack from outside your infrastructure, with timing on every check — so when a request hangs you already know which side stopped responding.

Start Your Free Trial →

Groq API Timeout Error: Why Requests Hang