How to Test and Mock the Groq API (2026 Guide)

The first integration test anyone writes against Groq calls the real endpoint, asserts the model said something sensible, and passes. It keeps passing for about three weeks. Then it fails on a Sunday because of a rate limit, fails again on a fork's pull request because the key is not there, fails a third time because the model phrased its answer differently — and someone adds `retry: 3` and stops reading the output.

The short version

Mock the network, not the SDK. Record fixtures from one real call rather than hand-writing them. Spend most of your testing effort on the failure responses, because those are the paths that only ever run during an incident. And keep one live check outside CI — a suite full of mocks cannot tell you Groq is down.

Why CI Should Not Call Groq Directly

None of these are theoretical. Each one is a specific way a live-calling test suite degrades into a suite nobody trusts.

Your build inherits their uptime. A Groq incident becomes a red build across every branch, and the team learns that red means "probably upstream" — which is exactly the habit that lets a real regression through.
Every pull request costs money. Token spend scales with contributor activity rather than with anything you would choose to pay for, and it is invisible until the invoice.
Secrets have to exist everywhere CI runs. That includes fork pull requests, where a real GROQ_API_KEY either fails to exist (so external contributors can never get a green build) or does exist (which is a far worse problem).
Model output is not deterministic. Asserting on generated prose produces failures that are not bugs, and the standard fix — loosening the assertion until it always passes — leaves a test that costs money and proves nothing.
Rate limits look exactly like flakiness. A parallelised suite trips Groq's limits precisely because it is fast, and the 429 arrives as an intermittent failure rather than as a clear message about concurrency.

The Four Layers, and What Each One Proves

Most teams have layer one and layer four missing, then wonder why a green suite did not prevent the outage. Each layer answers a different question, and no layer answers another layer's.

Layer	Runs	Proves	Cannot tell you
Unit / mocked	Every push	Your prompt construction, routing, parsing, retry and error-handling logic	Nothing about the provider itself
Recorded fixtures	Every push	That your code survives real response bodies, including error bodies	Whether those bodies are still current
Live smoke test	Pre-deploy, or nightly	Credentials work, model ids resolve, the contract still holds	Behaviour under real production load
Synthetic monitoring	Continuously	That the provider is up and fast right now, from outside your app	Nothing — this is the layer that pages you

The line worth internalising: layers one and two are about your code, layer three is about your assumptions, and layer four is about Groq. Adding more of the first two will never substitute for the last one.

A Mock You Can Paste In

Groq is OpenAI-compatible, and most codebases reach it through the OpenAI SDK pointed at Groq, or the `groq` package. Intercept at the HTTP boundary so the real client still runs — its validation, its error classes, its stream parsing all stay in the test.

import { setupServer } from 'msw/node';
import { http, HttpResponse } from 'msw';

const server = setupServer(
  http.post('https://api.groq.com/openai/v1/chat/completions', () =>
    HttpResponse.json(SUCCESS_FIXTURE)),
);

beforeAll(() => server.listen({ onUnhandledRequest: 'error' }));
afterEach(() => server.resetHandlers());
afterAll(() => server.close());

// onUnhandledRequest: 'error' is the important line. Without it, a call your
// code makes to an endpoint you forgot to stub goes to the real api.groq.com
// with whatever key is in the environment.

Get the success fixture by making one real call and saving the response verbatim, not by writing out what you think the body looks like. A hand-authored fixture tests your code against your own assumptions, which it will always pass.

Simulate the Four Failures That Actually Happen

This is where mocking earns its cost. Every one of these paths runs for the first time during an incident unless a test made it run earlier.

// 1. Rate limited — assert the backoff reads retry-after.
http.post('https://api.groq.com/openai/v1/chat/completions', () =>
  new HttpResponse(null, { status: 429, headers: { 'retry-after': '2' } }));

// 2. Server error — assert a bounded retry, then a clean surfaced failure.
http.post('https://api.groq.com/openai/v1/chat/completions', () =>
  new HttpResponse(null, { status: 500 }));

// 3. Timeout — assert your own deadline fires before the user's patience does.
http.post('https://api.groq.com/openai/v1/chat/completions', async () => {
  await new Promise((r) => setTimeout(r, 30_000));
  return HttpResponse.json(SUCCESS_FIXTURE);
});

// 4. Truncated stream — the one nobody tests, and the one that
//    quietly writes half an answer into your database as if it were whole.
http.post('https://api.groq.com/openai/v1/chat/completions', () => {
  const stream = new ReadableStream({
    start(controller) {
      controller.enqueue(new TextEncoder().encode(
        'data: {"choices":[{"delta":{"content":"The ans"}}]}\n\n'));
      controller.close(); // no [DONE], no finish_reason
    },
  });
  return new HttpResponse(stream, {
    headers: { 'Content-Type': 'text/event-stream' },
  });
});

The truncated stream is the expensive one

A stream that ends without a terminator is not an error at the HTTP layer — the status was 200 and bytes arrived. Code that accumulates deltas and writes the result on stream-close will happily store half an answer and mark it complete. Assert that a stream ending without a finish signal is treated as a failure, not as a short response.

The Groq Trap: Green Tests Hide the Only Thing You Bought Groq For

Groq is on your stack because it is fast. A mock returns in a microsecond, so every latency property of the system you are actually paying for is invisible to your test suite. A team can refactor from streaming to buffered, add a synchronous re-ranking hop, or move the call behind a queue, and the entire suite stays green while p95 time-to-first-token triples.

The second half of the same trap is rate limits. Groq's free and developer tiers are metered in tokens per minute, not just requests per minute, and a CI job that runs 40 integration tests against the live API with a 2,000-token system prompt will burn through TPM long before it burns through RPM. The failure looks like a flaky test — one job passes, the next 429s — which is exactly the shape that gets a retry loop bolted on instead of a mock.

So split the concerns explicitly: assert *correctness* against mocks, and assert *speed* against a real call that runs on a schedule rather than on every push. If a Groq response taking 8 seconds would be a production incident for you, then something has to be measuring that on real traffic, and it is not going to be your unit tests.

A latency assertion that belongs in a scheduled job, not in CI

// Runs nightly (or as a synthetic probe), NOT on every pull request.
const t0 = performance.now();
const res = await client.chat.completions.create({
  model: 'llama-3.3-70b-versatile',
  messages: [{ role: 'user', content: 'Reply with the single word: ok' }],
  stream: true,
});

let firstToken = 0;
for await (const chunk of res) {
  if (!firstToken && chunk.choices[0]?.delta?.content) {
    firstToken = performance.now() - t0;
  }
}

// Groq's value proposition IS this number. Alert on it.
if (firstToken > 1500) {
  throw new Error(`TTFT regressed: ${firstToken.toFixed(0)}ms`);
}

The One Test That Should Hit the Real API

Keep exactly one, keep it out of the pull-request path, and give it a narrow job: prove that the credentials work, that the model id still resolves, and that the response still has the shape your fixtures claim. It is the only thing standing between a fully mocked suite and a fixture that has quietly stopped describing reality.

// One test. Runs pre-deploy or nightly, never on a pull request.
// Its job is to prove the assumptions the mocks are built on.
const res = await fetch('https://api.groq.com/openai/v1/chat/completions', {
  method: 'POST',
  headers: {
    Authorization: `Bearer ${process.env.GROQ_API_KEY}`,
    'Content-Type': 'application/json',
  },
  body: JSON.stringify({
    model: 'llama-3.3-70b-versatile',
    messages: [{ role: 'user', content: 'Reply with the single word: ok' }],
    max_tokens: 5,
  }),
});

if (!res.ok) {
  throw new Error(`Groq smoke test failed: ${res.status}`);
}

// Assert the SHAPE the fixtures encode, not the words that came back.
const body = await res.json();
assertMatchesFixtureShape(body);

Run it on a schedule and before deploy. Do not run it on pull requests, because that reintroduces every problem in the first section — the secret exposure, the fork failures, the upstream-outage red builds — in exchange for a signal you already get nightly.

Where Testing Stops and Monitoring Starts

A test suite runs when you push. An outage happens when it happens. That gap is the entire reason a green pipeline and a broken product coexist so comfortably: nothing in CI is watching Groq at 02:00 on a Saturday, and a nightly smoke test tells you about a failure up to twenty-four hours after your users found it.

The layer that closes the gap is independent probing — real requests to api.groq.com on an interval, from outside your infrastructure, alerting on both errors and latency. It is also what answers the question your team will actually ask during an incident, which is never "did the tests pass" but always "is it us or is it them?"

Frequently Asked Questions

Should my CI pipeline call the live Groq API?

Almost never on every push. Live calls in CI make your build depend on someone else's uptime, cost money on every pull request, leak a real GROQ_API_KEY into every fork and pull-request runner, and produce non-deterministic failures that teams eventually silence by retrying. Keep the pull-request path fully mocked, and run a small live smoke test on a schedule or before deploy where a failure means something and a secret can be handled properly.

What is the difference between mocking the Groq SDK and mocking the HTTP layer?

Mocking the SDK replaces the client object, so your test never exercises the client's own validation, error classes or stream parsing — the parts most likely to change under you. Mocking the HTTP layer (msw, nock, a local server, or a base-URL override) intercepts the request to api.groq.com and lets the real client run against a fake network. The second is more work to set up once and catches an entire category of bug the first cannot see, including SDK upgrades that change behaviour silently.

How do I test what my code does when Groq returns a 429 or a 500?

Make the mock return them. This is the single highest-value thing a mock does, because the failure paths are the ones that are never exercised by hand and are always the ones that break in an incident. Return a 429 with a retry-after header and assert the backoff honours it rather than sleeping a hardcoded amount; return a 500 and assert the request is retried a bounded number of times; return a stream that ends mid-message and assert the partial output is not silently persisted as complete.

Do Groq mocks need to simulate token-per-minute limits?

They need to simulate the 429 that a TPM breach produces, yes — but they should not try to model the accounting. Groq returns a 429 with a retry-after when you exceed either requests or tokens per minute, and your code path is the same either way: back off, respect the header, surface it. What your test needs to prove is that the header is read and honoured rather than ignored in favour of a hardcoded sleep. Reproducing Groq's exact token bucket in a fixture is effort spent modelling someone else's implementation detail, and it will drift.

If everything is mocked, how do I know when Groq is actually down?

You do not, and that is the honest limit of a test suite: mocks tell you your code is correct, not that the provider is available. Availability is a monitoring question, not a testing one. Something outside your application has to be making real requests to api.groq.com on an interval and alerting when they fail or slow down — otherwise the first signal of an outage is a user, and the first thing your team will do is spend twenty minutes deciding whether the bug is yours.

Related Guides

Your Tests Cannot Page You at 2am

Mocks prove your code is right. They cannot tell you Groq stopped answering an hour ago. API Status Check probes Groq and the rest of your stack independently and alerts on errors and latency, so "is it us or is it them?" has an answer before anyone opens a terminal.

Start Your Free Trial →