Counting Cohere Tokens
Cohere gives you a real tokenize endpoint, which is more than most providers do — and then bills its most-used endpoint in a unit that is not tokens at all.
📡 Monitor your APIs — know when they go down before your users do
Better Stack checks uptime every 30 seconds with instant Slack, email & SMS alerts. Free tier available.
Affiliate link — we may earn a commission at no extra cost to you
Cohere is the provider in this set where the counting question splits three ways, because the platform is not one endpoint with one billing unit. Chat is token-denominated in the way you would expect. Embedding is token-denominated but deterministic, which changes the optimisation entirely. Rerank is not token-denominated at all — it is metered in search units, and a single call can consume many of them. A cost model that treats all three as “tokens” will be wrong about the one you probably call most.
Cohere also does something genuinely useful that most providers skip: it exposes tokenization as an API. You can ask the platform, authoritatively, how a given model splits a given string, without reimplementing a vocabulary and hoping you got it right. That is worth using — carefully, because it is a network call with a cost of its own.
The Tokenize Endpoint: Use It, But Not Per Request
Having a remote tokenizer removes an entire class of bug. On providers without one, you load a tokenizer artefact, hope it is the right version for the model you are calling, and discover months later that it was not. Cohere lets you settle the question directly, and the detokenize direction is equally useful when debugging why a truncation landed somewhere strange.
The trap is treating it as the counting mechanism for production traffic. It is a network round trip: it adds latency to every request you measure, and it consumes request allowance from the same account whose limits you are presumably trying to stay under. Counting every call this way means making two calls to send one, which is a strange trade when request rate is itself a constraint.
The right shape is a hybrid. Count locally with the model’s tokenizer for admission control on the hot path, and use the tokenize endpoint as ground truth: when you onboard a new model, when you change model versions, and on a small periodic sample of live traffic to confirm your local counts have not drifted. That gives you the accuracy of the remote counter and the latency of a local one, and it turns tokenizer drift into something you detect rather than something you eventually deduce.
Rerank Is Billed in Search Units, Not Tokens
This is the item most likely to break a Cohere cost forecast, because it violates the mental model imported from every other provider. Rerank is metered in search units, and the mapping from your request to units consumed is not one-to-one: documents longer than the unit length are split into multiple units, so a query against a set of long documents costs several units per call.
Two things follow. First, document length is a cost lever with the same weight as document count — sending fifty short passages and fifty long ones are not remotely the same request commercially, and chunking your corpus to sit near the unit boundary rather than sprawling across it is a real, structural saving. Second, and more importantly for capacity planning, your rerank spend scales with candidate-set size on every query. A retrieval pipeline that reranks a hundred candidates per query rather than twenty is five times the rerank cost for what is often a marginal quality difference. Measure that trade rather than assuming it.
The practical instruction is to keep a separate cost model per endpoint. One blended “Cohere spend” number, denominated in tokens, will mis-forecast whichever of chat, embed and rerank is dominant in your workload, and will point your optimisation effort at the wrong one. Our Cohere pricing guide covers the units in more detail.
Embeddings: Count Them, But Cache Them First
Embedding calls are token-denominated and subject to a per-input length limit, so counting matters both for cost and for avoiding rejections on oversized inputs. But before you tune the count, take the larger saving that is unique to this call type: embeddings are deterministic. The same model over the same input text yields the same vector every time, which means a second call for text you have already embedded is pure waste with no compensating benefit.
Key a persistent cache on model, input text and input type, and diff your corpus between indexing runs rather than re-embedding it wholesale. Most pipelines re-process everything on every run because that was simpler to write, and in a corpus that changes slowly this can be the overwhelming majority of the embedding volume — often the entire gap between the allowance you have and the one you were about to request.
One warning that costs teams real debugging time: invalidate the cache when you change embedding model. Vectors produced by different models are not comparable, and quietly mixing them corrupts retrieval quality without raising a single error. The symptom is search results that are subtly worse for reasons nobody can reproduce.
What Your Chat Count Misses
As on every provider, the assembled request is larger than the text you wrote. Role framing and structural tokens wrap each turn, which is why summing message string lengths under-counts and why the error grows with conversation length. Tool and connector definitions are serialised into context and charged on every request that carries them, including the many where nothing is invoked — and because they live in a separate field from the prose, they are the most commonly omitted item in any reconciliation.
Cohere adds one item the others do not: documents passed for grounding. If you supply retrieved passages with a chat request, those passages are input and are counted in full. This is the RAG-shaped equivalent of the retrieval overhead other grounded APIs incur server-side, except here it is entirely under your control — which means the size of your retrieval window is a token-cost decision you own, and passing twenty passages when five would answer the question is a choice with a visible price.
The completion is the remaining unknown. You cannot count it before it exists; you can only bound it. Set an explicit maximum you would genuinely accept and reserve prompt-plus-maximum against your budget, rather than leaving the output unbounded and hoping. The Cohere context window guide covers the per-request limit that this arithmetic has to satisfy.
Reconcile, and Know What Counting Cannot Tell You
Compare your pre-send estimate against the usage Cohere reports on a sample of live traffic, per endpoint rather than blended, and export the drift as a metric. A stable small gap is fine. A widening one means a system prompt grew, history is being included past where you thought it was truncated, a tool schema was added, a retrieval window widened, or a model version changed under you.
None of it tells you whether api.cohere.com is healthy. That distinction decides your response during an incident and it cannot be made from inside your own client, where a quota rejection and a provider outage produce the same picture: errors up, latency up, throughput down. One wants you to slow down; the other wants you to route away immediately. Independent external monitoring, probing on a schedule unrelated to your traffic, is the only thing that separates them in the moment rather than in the post-mortem. See our Is Cohere Down? guide and the Cohere rate limits guide.
Frequently Asked Questions
Does Cohere have a token counting endpoint?
Yes — Cohere is unusual in exposing tokenize and detokenize as first-class API calls, so you can get an authoritative count for a given model without reimplementing its vocabulary and hoping you matched the version. The detokenize direction is equally useful for debugging why a truncation landed somewhere unexpected. The catch is that it is a network call: it adds latency and it consumes request allowance from the same account whose limits you are trying to respect.
Should I call tokenize before every Cohere request?
No — that means making two calls to send one, which is self-defeating when request rate is itself one of the limits you are managing. Use a hybrid: count locally with the model’s tokenizer for admission control on the hot path, and call the tokenize endpoint as ground truth when you onboard a model, when you change model versions, and on a small periodic sample of live traffic. You get remote accuracy at local latency, and tokenizer drift becomes something you detect rather than something you eventually deduce from a billing surprise.
How is Cohere rerank billed if not in tokens?
In search units, and the mapping from your request to units consumed is not one-to-one: documents longer than the unit length are split, so a single query against a set of long documents costs several units. Two consequences follow. Document length is a cost lever with the same weight as document count, so chunking to sit near the unit boundary is a structural saving. And rerank spend scales with candidate-set size on every query — reranking a hundred candidates instead of twenty is five times the cost for what is often a marginal quality gain worth measuring rather than assuming.
Do I need to count tokens for Cohere embeddings?
Yes, for cost and for the per-input length limit that rejects oversized inputs. But take the bigger saving first: embeddings are deterministic, so the same model over the same text always yields the same vector and a repeat call is pure waste. Key a persistent cache on model, input and input type, and diff your corpus between indexing runs rather than re-embedding wholesale. In a slowly-changing corpus that is frequently the majority of embedding volume. Invalidate the cache when you change model — vectors from different models are not comparable, and mixing them degrades retrieval silently.
Why does my Cohere chat token count not match the sum of my messages?
Because the assembled request carries more than your message text. Role framing and structural tokens wrap every turn, so the under-count grows with conversation length. Tool and connector definitions are serialised into context and charged on each request that includes them, even when nothing is invoked. And any documents you pass for grounding are counted as input in full — which makes the size of your retrieval window a direct token-cost decision you control. Count the assembled request rather than summing strings, and the remainder is usually a system prompt or tool schema nobody remembered was there.
Can I use one cost model across Cohere chat, embed and rerank?
No, and trying to is the most common way Cohere forecasts go wrong. The three endpoints have genuinely different billing units and genuinely different traffic shapes: chat is token-denominated and relatively low frequency, embedding is token-denominated but deterministic and cacheable, and rerank is metered in search units that scale with both document length and candidate count. A blended token figure will mis-forecast whichever endpoint dominates your workload and will point your optimisation effort at the wrong one. Model and measure each separately.
My estimate and Cohere’s reported usage are diverging. What changed?
Track the drift per endpoint rather than blended, because a single averaged figure hides the one badly miscounted path behind several healthy ones. A stable small gap is expected. A widening one has five usual causes: a system prompt that grew by accretion, chat history included past the point you believed it was truncated, a tool or connector schema added by someone unaware it is charged per request, a retrieval window widened so more grounding documents ride along, or a model version change that brought a different vocabulary with it.
Related Guides
Over Budget, or Is Cohere Down?
Your own metrics cannot separate a quota rejection from an outage — both are errors and latency climbing, and they call for opposite responses. API Status Check probes api.cohere.com independently of your traffic, so you know whether to pace or to route around it.
Start Your Free Trial →🛠 Tools We Use & Recommend
Tested across our own infrastructure monitoring 200+ APIs daily
Uptime Monitoring & Incident Management
Used by 100,000+ websites
Monitors your APIs every 30 seconds. Instant alerts via Slack, email, SMS, and phone calls when something goes down.
“We use Better Stack to monitor every API on this site. It caught 23 outages last month before users reported them.”
Secrets Management & Developer Security
Trusted by 150,000+ businesses
Manage API keys, database passwords, and service tokens with CLI integration and automatic rotation.
“After covering dozens of outages caused by leaked credentials, we recommend every team use a secrets manager.”
SEO & Site Performance Monitoring
Used by 10M+ marketers
Track your site health, uptime, search rankings, and competitor movements from one dashboard.
“We use SEMrush to track how our API status pages rank and catch site health issues early.”