Cohere Batch and Embed Jobs: Embedding a Corpus Without Losing Half of It (2026)

Cohere is used most heavily for two things — embeddings and rerank — and both have a moment where you need to process far more than a single request’s worth of data at once: the first time you embed a knowledge base, or the full re-embed after you change models. For that, the async job path exists. You hand Cohere a dataset, receive a job, and retrieve the embeddings when it completes, without orchestrating thousands of synchronous calls against a rate limit.

The trap is specific to embeddings and worse than the general batch trap. Batch fails quietly everywhere — a job reaches completed with fewer outputs than inputs, no error raised, no metric moved. But for a chat completion, a missing row is a missing answer you might notice. For an embedding, a missing vector is a document that was never indexed, and it produces no error at query time. It just never appears in a search result. Your retrieval quietly develops blind spots, and nothing in your logs will ever point at them.

This guide is about the distance between the embed job finished and every document is retrievable. For a RAG system that distance is not a cost-accounting detail — it is the difference between a search index you can trust and one that silently omits whatever the batch happened to drop.

What the Async Job Actually Guarantees

Read these literally. The gap between what an embed job promises and what teams assume it promises is exactly the set of silent retrieval failures you will otherwise ship.

Property	What you actually get	What that forces you to build
Completeness	The job finishes; records can fail on their own	Reconcile returned ids against input ids. A completed job is not an indexed corpus.
Silence on failure	A missing vector raises no error at query time	Verify coverage explicitly; a dropped record is an invisible retrieval blind spot.
Ordering	Keyed by your id, not input order	Stable id on every record. Join vectors on it; matching by position corrupts search.
Latency	A completion window, not a scheduled time	Completion is an event. Keep the synchronous embed endpoint for query-time work.
Result window	Output retained for a limited period	Fetch and persist promptly into your vector store. The job is not your archive.

The mechanics are the standard batch contract; what makes Cohere’s case sharper is that the payload is embeddings, so the cost of an unreconciled job is not a visible gap but a search index that lies by omission.

The Lifecycle, and Where Each Stage Drops Documents

An embed job moves through four stages, and each has a failure that reports success while leaving documents unretrievable.

Prepare the dataset. You assemble the records to embed, each with an id you control. The failure is empty or malformed fields that will silently fail to embed, or duplicate ids that will collapse two documents into one vector. Validate locally: assert every record has content and a unique id before the job ever sees it.

Submit. You create the job and receive a job id. The failure is losing that id to a crash or a lost response, leaving a job running that nothing is tracking. Persist the job id transactionally the moment you have it, alongside the full set of record ids you submitted, so you know exactly which documents this job is responsible for making retrievable.

Wait. The job queues out of your sight. The failure is treating it as bounded — blocking on it, or re-submitting a merely-slow job and paying to embed the same corpus twice. Poll on a schedule and advance state from the authoritative job object.

Retrieve and reconcile. The job is done; you pull the embeddings and write them to your vector store. This is the stage that decides whether your search is trustworthy. Key the returned vectors by record id and compute the set difference against the ids you submitted. Every id in that gap is a document that will never be retrieved. Re-embed those — synchronously or in a follow-up job — before you call the corpus indexed. An unreconciled embed job is how a knowledge base ends up with holes nobody can see.

Why Position-Matching Corrupts Search Specifically

Carry a stable id on every record and join the returned vectors by that id. This matters more for embeddings than for any other batch payload, and here is the exact mechanism.

Suppose you match vectors to documents by position — the third vector is the embedding of the third input — and the job dropped record twelve. From record twelve onward, every vector is now attached to the wrong document. Document thirteen carries document fourteen’s meaning, and so on down the file. Nothing errors. The index builds cleanly. Then a user searches, and the system confidently returns documents whose vectors say they are relevant while their text is about something else entirely. It is not a crash and not an empty result — it is fluent, plausible wrongness, the hardest kind of bug to notice and the slowest to trace. A stable id makes the drop a visible gap instead of a silent misalignment.

What to Instrument

Five numbers make an embed job legible: the age of your oldest non-terminal job against the window you expected; the submitted-versus-returned record count per job; the count of records re-embedded because the job dropped them; the count of documents in your vector store versus documents in your source of truth, checked after every job; and the count of jobs re-submitted because a slow one was mistaken for a dead one. The document-count reconciliation is the one that catches the silent retrieval blind spots — it is the only check that notices a corpus is missing vectors before a user does.

What none of them can tell you is whether api.cohere.com was degraded while your job sat queued — that failure is invisible from inside a system that was waiting rather than calling, and it is exactly the gap external monitoring closes.

Frequently Asked Questions

When should I use Cohere Embed Jobs instead of the synchronous embed endpoint?

Use the async job path when the corpus is large enough that streaming it through the synchronous endpoint would take hours of your own orchestration and hit rate limits the whole way. Embed Jobs takes a dataset and returns the embeddings as a job you retrieve later, which is the right shape for the initial embedding of a knowledge base or a periodic full re-embed after a model change. Keep the synchronous endpoint for query-time embedding and for any single document a user is waiting on -- the job path trades immediacy for throughput, and that is a bad trade when a request is blocked on the answer.

My Cohere Embed Job finished but the output has fewer vectors than my dataset. Is that normal?

Yes, and it is the failure mode that quietly breaks vector search. A job reaching a terminal completed state means the job is done, not that every input produced an embedding. Individual records fail independently -- an empty field, a record over a length limit, a transient error on one row -- and the job still reports success with those records absent. For embeddings this is especially dangerous because a missing vector is not an error at query time; it is simply a document that can never be retrieved, silently absent from every search result. Reconcile the returned ids against the input ids and re-embed the gap before you consider the corpus indexed.

Do embeddings come back in the same order as my input records?

Do not depend on it. Results are keyed by the identifier attached to each record, and with a large job they can return in a different order or with rows missing. For embeddings, matching by position is catastrophic in a specific way: if the first missing record shifts the alignment, you attach the wrong vector to every subsequent document, and your search returns confident, plausible, completely incorrect results with no error anywhere. Carry a stable id on every record and join the vectors back by that id, never by row position.

How long does a Cohere Embed Job take?

It depends on the size of the dataset and on queue depth you cannot see, and you get a completion that arrives within a window rather than at a time you can schedule. A large corpus can take a while, and the same job can finish faster or slower run to run. Treat completion as an event you react to -- poll the job, and when it is done, retrieve and reconcile -- rather than a deadline you build a user-facing flow around. Nothing that a person is waiting on should sit behind an embed job.

Should I poll the embed job or wait for a notification?

Poll, and make the poll the thing that actually advances the job in your system. A scheduled sweep that lists non-terminal jobs and reads their status from the API survives a dropped notification, a receiver that was deploying, and a job submitted by a different process than the one waiting. Build that first and make advancing to done idempotent so a duplicate signal changes nothing. Any push notification is then just an accelerator that lets the poll find the answer sooner, not a channel you depend on.

How do I tell a slow embed job from a Cohere outage?

From inside your own telemetry you often cannot, because a job stalled behind a Cohere incident and a job merely far back in a healthy queue both look identical -- not done, no error, no latency metric moving, because you are waiting rather than calling. The internal signal is the age of your oldest non-terminal job measured against the window you expected. When it crosses that line, treat the job as stuck rather than slow, and lean on independent monitoring of api.cohere.com to distinguish a provider incident from an ordinary long queue.

Related Guides

A Stalled Embed Job Looks Exactly Like a Quiet Day

When the async path degrades, your telemetry goes quiet rather than red — no errors, no latency spike, just a job that never reaches done and a corpus that never finishes indexing. API Status Check probes api.cohere.com independently and alerts on errors and latency, so you find out from a monitor instead of from a search that quietly stopped returning half your documents.

Start Your Free Trial →