How to Migrate from OpenAI to Cohere: Complete API Migration Guide (2026)
Updated July 2026 · 9 min read · API Status Check
Quick Answer
Cohere runs its own v2 API at api.cohere.com/v2 with its own SDK and its own request shape. This is the least drop-in migration of the major OpenAI alternatives: the chat payload, the RAG surface, and the response object all differ, so plan a real adapter layer rather than a base_url swap.
📡 Monitor your APIs — know when they go down before your users do
Better Stack checks uptime every 30 seconds with instant Slack, email & SMS alerts. Free tier available.
Affiliate link — we may earn a commission at no extra cost to you
Why Teams Migrate from OpenAI to Cohere
Cohere is an enterprise-retrieval company more than a chat company. Teams migrate for the pieces OpenAI does not offer as first-class primitives: a dedicated Rerank endpoint, multilingual Embed v3 with compression, and grounded generation with built-in citations. The usual pattern is keeping OpenAI for generation and moving the retrieval stack to Cohere.
Migrate selectively. A full replacement is rarely the right call. Keep image generation, audio, and general open-ended chat — the strongest case for Cohere is the retrieval layer (Embed + Rerank), not replacing generation wholesale on OpenAI, and move the call paths where Cohere is genuinely better. Because you end up running two providers, an independent monitor on both endpoints stops a provider incident from looking like an application bug.
Step 1: Swap the Client
Start with the smallest change that produces a real response, then fix parameters and models from actual errors rather than guessing up front.
# Cohere v2 has its own shape. Write an adapter, not a base_url swap.
import cohere
co = cohere.ClientV2(api_key=os.environ["COHERE_API_KEY"])
# Chat — messages look similar, the client and response object do not
resp = co.chat(
model="command-a-03-2025",
messages=[{"role": "user", "content": "hi"}],
)
text = resp.message.content[0].text
# Embeddings — input_type is required and has no OpenAI equivalent.
# Use search_document for your corpus, search_query at query time.
docs = co.embed(
model="embed-v4.0",
texts=["chunk one", "chunk two"],
input_type="search_document",
)
# Rerank has no OpenAI counterpart — this is usually the biggest win.
ranked = co.rerank(
model="rerank-v3.5",
query="how do I rotate an api key?",
documents=[d["text"] for d in candidates],
top_n=5,
)Step 2: Map Your Models
There is no exact equivalence between OpenAI models and Cohere models. Treat this as a starting point and validate each swap against your own evaluation set before cutting traffic over.
gpt-4o→command-a-03-2025Cohere's flagship chat/RAG model. Strong on grounded enterprise answering; evaluate on your own data before assuming parity on open-ended tasks.
gpt-4o-mini→command-r7bSmall, cheap tier for routing and extraction.
text-embedding-3-small→embed-v4.0The strongest reason to migrate. Multilingual, supports input_type-aware embedding and compressed vectors — but a full corpus re-embed is mandatory.
(no OpenAI equivalent)→rerank-v3.5A dedicated reranking endpoint. Replacing a home-rolled LLM reranker with Rerank is often the single highest-ROI part of a Cohere migration.
dall-e-3 / whisper-1→— (not offered)No image generation or audio. Those calls stay on OpenAI.
Monitor both providers during the cutover
Running OpenAI and Cohere side by side means two dependencies that can fail independently. Better Stack checks both endpoints from multiple regions and alerts you the moment either degrades. Free tier included.
Try Better Stack Free →Step 3: Fix the Parameter Differences
This is where most migrations actually break. The client swap takes minutes; these differences take a day.
Different request shape, not different values
Cohere v2 uses its own chat payload and response object. Do not try to forward an OpenAI payload — write a thin adapter that maps your internal message type to each provider. Teams that skip the adapter end up with provider details leaking through the whole codebase.
input_type on embeddings is required
Cohere's Embed expects an input_type such as search_document or search_query, and using the wrong one measurably degrades retrieval quality. There is no OpenAI analogue, so it is easy to omit and hard to notice.
Documents are a first-class field
Instead of stuffing retrieved context into a system prompt, you pass documents and get grounded output with citation spans. That is a better RAG contract, but it means rewriting your prompt-assembly code rather than reusing it.
Embed dimensions and compression
embed-v4.0 vectors differ from OpenAI's, and the compressed variants differ again. Pick one configuration before backfilling — changing it later means re-embedding the corpus a second time.
Step 4: Know the Errors You Will Hit
These are the failures that show up in the first week after pointing production traffic at Cohere:
"HTTP 400 invalid request on migrated payloads"The signature of trying to send an OpenAI-shaped body to api.cohere.com/v2. Build the Cohere payload explicitly."Retrieval quality drops after migrating embeddings"Usually a missing or wrong input_type, or mixing embed-v4.0 vectors with older OpenAI vectors in one index. Both are silent — no error, just worse results."HTTP 429 Too Many Requests"Trial keys are heavily limited and are frequently mistaken for an outage during evaluation. Confirm the key tier in dashboard.cohere.com before debugging."HTTP 503 / elevated latency"Check status.cohere.com. Cohere publishes component-level status, so confirm whether Chat, Embed, or Rerank specifically is affected — a Rerank incident does not imply Chat is down.Step 5: Keep OpenAI Wired as a Fallback
Do not delete the OpenAI path. The most resilient production setup after a migration is a circuit breaker that fails back automatically, so a Cohere incident degrades quality instead of taking the feature down.
# Circuit breaker: Cohere primary, OpenAI fallback
import time
FAILS = {"count": 0, "open_until": 0.0}
THRESHOLD, COOLDOWN = 3, 300 # 3 errors -> skip primary for 5 minutes
def complete(messages):
if time.time() < FAILS["open_until"]:
return call_openai(messages) # breaker open
try:
out = call_cohere(messages)
FAILS["count"] = 0
return out
except Exception:
FAILS["count"] += 1
if FAILS["count"] >= THRESHOLD:
FAILS["open_until"] = time.time() + COOLDOWN
alert("Cohere breaker opened — serving from OpenAI")
return call_openai(messages)
# Alert on fallback RATE, not just on errors. A migration that silently
# serves 60% of traffic from the fallback is an outage you never noticed.Cutover Checklist
Before you route traffic
- Run your eval set against both providers on identical inputs
- Move model IDs into config so a deprecation is not a deploy
- Load-test against Cohere's real rate limits, not OpenAI's
- Compare p95 latency, not just average, on your hottest path
- Put the migration behind a percentage flag you can roll back
After you route traffic
- Track error rate and latency per provider, separately
- Alert on fallback activation rate — silent failover hides outages
- Subscribe to status.cohere.com and keep an independent check running
- Watch cost per successful request, not cost per token
- Re-test your fallback path monthly; an untested fallback is not one
Frequently Asked Questions
Is Cohere a drop-in replacement for OpenAI?
No, and it is the clearest non-drop-in of the major alternatives. Cohere's v2 API has its own request shape, its own response object, and a documents field instead of context stuffed into a prompt. Budget for a small adapter layer that maps your internal message type onto each provider rather than attempting a base_url swap.
What should I actually migrate to Cohere?
The retrieval stack. Embed v4 for multilingual and compressed vectors, and Rerank for precision on top of a cheap first-stage retriever. Many teams keep OpenAI for generation and move only Embed and Rerank — that captures most of the benefit for a fraction of the migration risk.
Do I have to re-embed everything to use Cohere Embed?
Yes. Cohere's vectors are a different space and dimensionality than OpenAI's text-embedding-3-* models, and the compressed variants differ again. Choose your final configuration first, dual-write into a new index, backfill, then cut reads over — re-embedding twice is the common and avoidable mistake.
Why did my search results get worse after migrating to Cohere embeddings?
Almost always a missing or incorrect input_type. Cohere expects search_document when embedding your corpus and search_query at query time; using one value for both silently degrades relevance with no error. The other cause is a partially backfilled index mixing old OpenAI vectors with new Cohere ones.
How do I monitor a Cohere migration?
Track Chat, Embed, and Rerank as separate dependencies, because Cohere reports and fails at component level — a Rerank incident does not mean Chat is down. Watch error rate and p95 latency per endpoint and run an independent synthetic check, since your own metrics will show degradation before a status page confirms it.
Related Guides
Alert Pro
14-day free trialStop checking — get alerted instantly
Next time Cohere goes down, you'll know in under 60 seconds — not when your users start complaining.
- Email alerts for Cohere + 9 more APIs
- $0 due today for trial
- Cancel anytime — $9/mo after trial