How to Test and Mock the Cohere API

The first integration test anyone writes against Cohere calls the real endpoint, asserts the model said something sensible, and passes. It keeps passing for about three weeks. Then it fails on a Sunday because of a rate limit, fails again on a fork's pull request because the key is not there, fails a third time because the model phrased its answer differently — and someone adds `retry: 3` and stops reading the output.

Why CI Should Not Call Cohere Directly

None of these are theoretical. Each one is a specific way a live-calling test suite degrades into a suite nobody trusts.

• Your build inherits their uptime. A Cohere incident becomes a red build across every branch, and the team learns that red means "probably upstream" — which is exactly the habit that lets a real regression through.
• Every pull request costs money. Token spend scales with contributor activity rather than with anything you would choose to pay for, and it is invisible until the invoice.
• Secrets have to exist everywhere CI runs. That includes fork pull requests, where a real CO_API_KEY either fails to exist (so external contributors can never get a green build) or does exist (which is a far worse problem).
• Model output is not deterministic. Asserting on generated prose produces failures that are not bugs, and the standard fix — loosening the assertion until it always passes — leaves a test that costs money and proves nothing.
• Rate limits look exactly like flakiness. A parallelised suite trips Cohere's limits precisely because it is fast, and the 429 arrives as an intermittent failure rather than as a clear message about concurrency.

The Four Layers, and What Each One Proves

Most teams have layer one and layer four missing, then wonder why a green suite did not prevent the outage. Each layer answers a different question, and no layer answers another layer's.

The line worth internalising: layers one and two are about your code, layer three is about your assumptions, and layer four is about Cohere. Adding more of the first two will never substitute for the last one.

A Mock You Can Paste In

Cohere is its own — not OpenAI-shaped, and most codebases reach it through the `cohere-ai` client. Intercept at the HTTP boundary so the real client still runs — its validation, its error classes, its stream parsing all stay in the test.

import { setupServer } from 'msw/node';
import { http, HttpResponse } from 'msw';

const server = setupServer(
  http.post('https://api.cohere.com/v2/chat', () =>
    HttpResponse.json(SUCCESS_FIXTURE)),
);

beforeAll(() => server.listen({ onUnhandledRequest: 'error' }));
afterEach(() => server.resetHandlers());
afterAll(() => server.close());

// onUnhandledRequest: 'error' is the important line. Without it, a call your
// code makes to an endpoint you forgot to stub goes to the real api.cohere.com
// with whatever key is in the environment.

Get the success fixture by making one real call and saving the response verbatim, not by writing out what you think the body looks like. A hand-authored fixture tests your code against your own assumptions, which it will always pass.

Simulate the Four Failures That Actually Happen

This is where mocking earns its cost. Every one of these paths runs for the first time during an incident unless a test made it run earlier.

// 1. Rate limited — assert the backoff reads retry-after.
http.post('https://api.cohere.com/v2/chat', () =>
  new HttpResponse(null, { status: 429, headers: { 'retry-after': '2' } }));

// 2. Server error — assert a bounded retry, then a clean surfaced failure.
http.post('https://api.cohere.com/v2/chat', () =>
  new HttpResponse(null, { status: 500 }));

// 3. Timeout — assert your own deadline fires before the user's patience does.
http.post('https://api.cohere.com/v2/chat', async () => {
  await new Promise((r) => setTimeout(r, 30_000));
  return HttpResponse.json(SUCCESS_FIXTURE);
});

// 4. Truncated stream — the one nobody tests, and the one that
//    quietly writes half an answer into your database as if it were whole.
http.post('https://api.cohere.com/v2/chat', () => {
  const stream = new ReadableStream({
    start(controller) {
      controller.enqueue(new TextEncoder().encode(
        'data: {"choices":[{"delta":{"content":"The ans"}}]}\n\n'));
      controller.close(); // no [DONE], no finish_reason
    },
  });
  return new HttpResponse(stream, {
    headers: { 'Content-Type': 'text/event-stream' },
  });
});

The Cohere Trap: A Fixture Recorded Against the Wrong API Version

Cohere's chat API is not OpenAI-shaped, and it has more than one generation in active use. The v1 and v2 chat surfaces differ in where the message goes, how the assistant's reply is nested, and what an error body looks like. A fixture copied from a blog post, an old integration test, or an LLM's memory of the SDK will frequently be the wrong generation — and because a mock never validates itself, that mismatch survives right up until a real call is made.

This makes the recording step matter more here than anywhere else on this list. Do not hand-write a Cohere response body. Make one real call with the client version you actually have installed, save the response verbatim, and use that as the fixture. Hand-authored fixtures encode what you believed the API looked like; recorded ones encode what it is.

The same applies to errors. Cohere's error payloads do not mirror the OpenAI error envelope, so if your handler was written by pattern-matching a generic `error.error.message` path, a hand-written fixture that happens to use that shape will validate a handler that cannot read a real Cohere failure. Record a genuine 401 and a genuine 429 once, keep them, and let the handler be tested against the real thing.

Record the fixture instead of writing it

// scripts/record-cohere-fixture.ts — run once, by hand, with a real key.
import { CohereClientV2 } from 'cohere-ai';
import { writeFileSync } from 'node:fs';

const cohere = new CohereClientV2({ token: process.env.CO_API_KEY });

const res = await cohere.chat({
  model: 'command-a-03-2025',
  messages: [{ role: 'user', content: 'Reply with the single word: ok' }],
});

// Verbatim. Do not tidy it, do not reshape it into OpenAI's envelope.
writeFileSync(
  'test/fixtures/cohere-chat-success.json',
  JSON.stringify(res, null, 2),
);

// Repeat once with a deliberately bad token to capture the real 401 body.

The One Test That Should Hit the Real API

Keep exactly one, keep it out of the pull-request path, and give it a narrow job: prove that the credentials work, that the model id still resolves, and that the response still has the shape your fixtures claim. It is the only thing standing between a fully mocked suite and a fixture that has quietly stopped describing reality.

// One test. Runs pre-deploy or nightly, never on a pull request.
// Its job is to prove the assumptions the mocks are built on.
const res = await fetch('https://api.cohere.com/v2/chat', {
  method: 'POST',
  headers: {
    Authorization: `Bearer ${process.env.CO_API_KEY}`,
    'Content-Type': 'application/json',
  },
  body: JSON.stringify({
    model: 'command-a-03-2025',
    messages: [{ role: 'user', content: 'Reply with the single word: ok' }],
    max_tokens: 5,
  }),
});

if (!res.ok) {
  throw new Error(`Cohere smoke test failed: ${res.status}`);
}

// Assert the SHAPE the fixtures encode, not the words that came back.
const body = await res.json();
assertMatchesFixtureShape(body);

Run it on a schedule and before deploy. Do not run it on pull requests, because that reintroduces every problem in the first section — the secret exposure, the fork failures, the upstream-outage red builds — in exchange for a signal you already get nightly.

Where Testing Stops and Monitoring Starts

A test suite runs when you push. An outage happens when it happens. That gap is the entire reason a green pipeline and a broken product coexist so comfortably: nothing in CI is watching Cohere at 02:00 on a Saturday, and a nightly smoke test tells you about a failure up to twenty-four hours after your users found it.

The layer that closes the gap is independent probing — real requests to api.cohere.com on an interval, from outside your infrastructure, alerting on both errors and latency. It is also what answers the question your team will actually ask during an incident, which is never "did the tests pass" but always "is it us or is it them?"

Frequently Asked Questions

Related Guides

• Cohere API Error Codes Explained
• Cohere API Rate Limits Explained
• Cohere API Key Setup Guide
• Cohere API Streaming Errors
• Cohere API Failover Guide
• Cohere API Monitoring Guide
• Is Cohere Down? Outage Checking Guide
• Live Cohere Status Check