Groq API Connection Reset: Why the Socket Dies Between Requests

Groq returns most completions in well under a second. That speed is the reason this error shows up here more than it does on slower providers, and it is the reason the usual advice — raise your timeout, add retries — does nothing for it.

A connection reset means the peer sent a TCP RST. Your request did not time out and it did not get a status code; the socket was torn down mid-flight. In Node this surfaces as ECONNRESET or the friendlier “socket hang up”, in Python as a ConnectionResetError wrapped by httpx or the OpenAI SDK as APIConnectionError, and in Go as an unexpected EOF. All four are the same event at different altitudes.

Why Groq gets more resets than a slow provider does

Every HTTP client you are likely to use keeps connections open after a response so the next request can skip DNS, TCP and TLS. The server keeps its own idle timer on the same socket, and when the two disagree you get a reset: your client picks a socket out of the pool, writes a request onto it, and only then discovers the far end closed it some seconds ago. The RST arrives after the write, so it looks like the request failed rather than the connection.

Groq makes that disagreement more likely for a structural reason. The window in which a socket is idle is the gap between requests minus the duration of a request. When a provider takes fifteen seconds per call, sockets spend most of their life busy. When Groq answers in 600 milliseconds, the same traffic pattern leaves the same socket idle for almost the entire interval — so a far greater share of your pool is sitting past the server’s idle threshold at any moment.

That is why this error clusters in the shapes it does: the first request after a quiet period, the first request after a serverless function warm-starts onto a reused container, and burst traffic against a pool that was sized for steady load. It is almost never correlated with a Groq incident, which is exactly why checking the status page first wastes the first twenty minutes.

The OpenAI SDK hides the first two resets from you

Groq exposes an OpenAI-compatible surface, so most teams point the official OpenAI SDK at it. That SDK retries connection errors internally — twice by default — before raising anything. The APIConnectionError your code finally catches is a summary of three attempts, not one, and the two you never saw already consumed wall-clock time inside a function that may have its own duration cap.

This matters twice over. It means your observed reset rate is understating the real one by roughly a factor of three, so the pool problem looks smaller than it is. And it means a reset storm burns three times the latency budget you think it does, which is how an ECONNRESET problem gets misdiagnosed as a timeout problem on a serverless platform.

Set maxRetries: 0 temporarily while you are diagnosing, log the raw cause, and you will usually find the true failure rate is high enough to make the fix obvious.

Fix it in the pool, not in the retry loop

The durable fix is to make sure your client never hands out a socket the server has already given up on. Set your own idle timeout comfortably below the server’s, cap how long any single connection lives, and let the pool churn.

// Node 20+ — undici pool tuned so sockets never outlive the server's idle window.
import { Agent, setGlobalDispatcher } from 'undici';

setGlobalDispatcher(new Agent({
  keepAliveTimeout: 4_000,        // retire idle sockets well before the edge does
  keepAliveMaxTimeout: 10_000,    // hard ceiling regardless of server hints
  connections: 32,                // enough headroom that bursts do not queue
}));

// Python: httpx.Client(limits=httpx.Limits(keepalive_expiry=4.0))

Then handle the failure itself. The rule that keeps retries honest is the same everywhere: retry only when nothing was read back, cap the attempts, and always add jitter — a reset that hits many sockets at once will otherwise synchronise every client into a single reconnect wave that re-fails together.

// Retry a reset ONLY when nothing was written back to you.
async function callGroq(body: unknown, attempt = 0): Promise<Response> {
  try {
    const res = await fetch('https://api.groq.com/openai/v1/chat/completions', {
      method: 'POST',
      headers: { Authorization: `Bearer ${process.env.GROQ_API_KEY}`, 'Content-Type': 'application/json' },
      body: JSON.stringify(body),
    });
    if (!res.ok) throw new Error(`upstream ${res.status}`);
    return res;
  } catch (err) {
    const code = (err as NodeJS.ErrnoException)?.cause ?? err;
    const reset = String(code).includes('ECONNRESET') || String(code).includes('socket hang up');
    if (reset && attempt < 2) {
      // Zero bytes received: safe to retry, and near-instant on a fresh socket.
      await new Promise((r) => setTimeout(r, 50 + Math.random() * 100));
      return callGroq(body, attempt + 1);
    }
    throw err;
  }
}

When it is not the pool

Two other causes produce the identical error and need different fixes. A corporate proxy or TLS-inspecting middlebox that does not like the connection will send an RST during or just after the handshake — the tell is that it fails on a genuinely fresh connection, from one network only, and reproduces under curl.

The second is a reset during the request body upload rather than after it. Very large prompts pushed through an intermediary with a body-size limit get cut off while you are still writing, which surfaces as EPIPE on some clients and ECONNRESET on others. If your resets correlate with prompt length rather than with idle time, that is the one you have.

Neither of those is fixed by pool tuning, which is why the idle-versus-fresh distinction in the triage box is the first thing to establish.

Frequently Asked Questions

Related Groq Guides

• Is Groq Down? Outage Checking Guide
• Groq API Timeout Error: Why Requests Hang
• Groq API Error Codes Reference
• Why Your Groq API Stream Breaks
• Groq API Retry Budget Guide
• Groq API Failover Guide
• Groq API Monitoring Guide