Leaving Together AI: The Vendor Exit Guide for Teams Already in Production (2026)

Ask an engineering team how hard it would be to move off Together AI and the answer is usually a confident one, because the question they hear is about code. Changing a base URL, a client library and an environment variable really is an afternoon. The reason exits routinely take a quarter instead is that the code was never the thing holding you in place. What holds you in place is the set of assets you created inside the vendor rather than merely stored there, and the behaviour of your product that was tuned, usually without anyone writing it down, to the specific characteristics of one model.

Leaving because of an outage today? That is failover, not exit — check live Together AI status first, then read the Together AI failover guide. Permanent migrations decided during an incident are the ones that get half-finished.

Exit is not failover, and it is not shopping

Three different decisions get discussed with the same vocabulary, and conflating them is why exit plans are so often a list of alternatives with no dates on it. Separating them first makes the rest of this page usable.

Decision	Question it answers	Reversible?
Failover	Where does traffic go for the next twenty minutes?	Yes — that is the point
Fallback ranking	Is the substitute's answer worth the same?	Yes, per request
Vendor selection	Which provider should we be using at all?	Cheap before you integrate
Exit	What do we lose when the account closes?	No — assets and access end

A working failover path is often presented as proof that an exit is easy. It is weak evidence, because failover is designed to preserve everything an exit has to replace: the same prompts, the same schemas, the same index, the same baselines, and the assumption that the original provider comes back. See the Together AI fallback ranking guide for why a substitute that keeps the lights on is not necessarily a substitute you can live on.

Seven assets you built inside the vendor

The useful split is not between data and code. It is between things you stored at the provider, which come back, and things you derived from the provider, which do not — because the thing that produced them is the thing you are leaving.

Asset	Stored or derived?	What replacing it costs
Keys, org config, rate-limit tiers	Stored	Hours — plus a quota request with lead time on the new side
Prompts and templates	Stored in your repo, tuned to one model	The text ports; the behaviour does not. Re-tune and re-baseline
Fine-tuned or adapted checkpoints	Derived, vendor-hosted	Often not exportable — a re-training budget line, not a ticket
Uploaded documents and file stores	Stored	Re-upload and re-parse; also a deletion obligation on the way out
Embeddings and the vector index	Derived — model-specific by construction	Full re-embed and re-index; scales with corpus size, not traffic
Evaluation baselines and golden-set scores	Derived	Scores are not comparable across providers; re-baseline before you judge
Request logs, usage history, invoices	Stored — often only in their console	Export before notice; access ends before your need for them does

Together AI is where the stored-versus-derived distinction earns its keep, because the platform is most often adopted precisely to create a derived asset — an adapted or fine-tuned model — and that is the one row in the table with a four-figure replacement cost and a lead time measured in weeks.

📡

Recommended

Watch Both Providers During the Cutover

A migration window is the one period where you are paying two vendors and can least afford to learn about a degradation from a customer. External monitoring against both endpoints tells you which side moved before the comparison data is polluted.

Try Better Stack Free →

The exit-readiness test you can run while you are still happy

None of this is worth doing as a project when you are leaving; all of it is worth having as a property of the integration. Each item below is also an availability control, which is the argument that gets it prioritised: the same four things carry you through an outage, a price change and a model deprecation.

# EXIT READINESS -- five checks, none of which require leaving.
#
# 1. ONE CONSTRUCTION POINT
#    grep the repo for the base URL and the client constructor.
#    More than one hit = the migration has an unknown surface area.
#
# 2. PROVENANCE ON EVERY STORED RESULT
#    provider, model id, sdk version, prompt hash, rung id.
#    Without these you cannot tell, after the cutover, whether an
#    old record was produced by the system you still operate.
#
# 3. A CURRENT EVAL BASELINE
#    A golden set with scores from THIS WEEK. A baseline measured
#    only after you start migrating cannot separate the provider
#    change from every other change shipped that month.
#
# 4. YOUR OWN REQUEST LEDGER
#    Written on your side at call time. The vendor console is not
#    a system of record you control; it closes with the account.
#
# 5. AN ANNUAL DRILL, NOT A DOCUMENT
#    Route one real feature to a second provider for one day and
#    record what broke. A written plan that has never been run is
#    an estimate; the drill is the only thing that produces a date.
#
# The number you want on a page somewhere: cost and elapsed time to
# rebuild the DERIVED assets. That figure is what a renewal
# negotiation is actually about.

Item two is the one skipped most often and the one that hurts longest. Records written before a migration and records written after it are indistinguishable without provenance fields, which means every retrospective quality question — did the new provider make this worse? — becomes unanswerable exactly when it is being asked. The Together AI eval regression guide covers the fields in detail; the exit is the case that punishes their absence hardest.

The cutover: shadow, ramp by feature, then notice

A shadow window — the new provider answering the same production requests without those answers reaching users — is the only inexpensive way to compare quality on traffic that is genuinely yours. It costs real money on both sides, so bound it by date rather than by confidence, and use the time to collect paired results rather than to watch a dashboard.

Ramp by feature, not by traffic percentage. A percentage split across a mixed workload averages a badly regressed use case together with four healthy ones, and the aggregate stays green while the one that matters is broken. Start with the feature where a poor answer costs least, and keep the rung or provider identifier on every stored result so the comparison survives the window. Where you have per-tenant obligations, read the Together AI multi-tenant guide before assuming a single global switch is acceptable.

The step with a hard external deadline is the commercial one. Notice periods and renewal dates do not wait for the migration to feel finished, and an exit that is technically complete but was never formally given notice simply renews. Put the notice date on the same plan as the code, work backwards from it, and treat the export of logs, invoices and usage history as a task that must land before that date rather than after it.

The Together AI trap: whether your checkpoint can leave is a question to answer before you train it

Two things make a Together AI exit different, and they pull in opposite directions. The first is that much of the catalogue is open-weight, which makes the exit look free: the same model is available from other hosts, so surely you just point elsewhere. That reasoning holds at the licence level and fails at the evaluation level, because a host controls the serving stack, the quantisation, the context handling and the endpoint class, and the “same” model served differently is a different model for any purpose you would actually measure. Treat a same-model host change as a full re-baseline, not as a configuration edit. The second is the opposite problem: if you fine-tuned or adapted a model on the platform, that checkpoint is a derived asset created inside the vendor, and whether you can take the weights with you is a term to establish at the start of the fine-tuning project rather than at the end of the relationship. If the answer is no, then the honest line in your exit plan is a re-training budget with a schedule and a data dependency, not a migration ticket — and the training data you used had better still exist in a form you can rebuild from, which is a question that catches teams who treated the upload as the archive. There is also an operational wrinkle: because incidents here are frequently scoped to individual models rather than the platform, a shadow window that compares only aggregate health can attribute a model-specific degradation to the migration itself.

Six exits that look finished and are not

Every row below passes the check most teams use to declare the migration done — production traffic to the old provider is zero. None of them is finished.

What it looks like	What is still coupled	How you find it
Web traffic is 100% on the new provider	A nightly batch job, a queue worker or a data pipeline still holds an old key	Watch the old key's usage for a full billing cycle, not a day
Keys revoked, account cancelled	Customer documents still sit in a vendor-side file store — an unmet deletion obligation	Enumerate uploads before revoking access, and get deletion in writing
Search still works perfectly	The index is built from the old provider's embeddings; quality is anchored to a vendor you left	Store the embedding model id on every vector and query for stragglers
Prompts ported verbatim, no errors	Output quality moved and nobody had a baseline to see it move	Paired golden-set comparison, run before and after, on the same items
Error rate unchanged after cutover	Timeouts, loading states and concurrency budgets were tuned to the old latency profile	Compare p95 and in-flight concurrency, not availability
Finance says the bill went to zero	The agreement auto-renewed because notice was never formally given	A calendar entry on the notice date, owned by a person, not a doc

Rows three and five are the ones that survive longest, because both are invisible to every monitor you already run: a stale index returns results and a slower provider returns answers. The signals that would catch them — an embedding model id stored beside each vector, and a latency comparison rather than an availability one — have to be in place before the cutover to be worth anything during it.

What you owe other people on the way out

An exit is the one integration event with obligations pointing outward. If Together AI appears on a subprocessor list, in a customer security questionnaire or in a data-processing agreement, removing it is part of the migration and not paperwork that follows it. The same is true in reverse for whoever answers next.

Deletion, confirmed in writing. Uploaded files, fine-tuning data and anything retained in logs. Ask while the account is open; see the Together AI data privacy guide for what is typically retained and for how long.
Subprocessor and disclosure updates. Customers who were told which provider processes their data are owed the correction, and a stale list is a finding in the next review.
Historical results that name a model. Anything you surfaced to users as produced by a specific model stays on record; decide whether those results are archived, re-run or labelled.
Your own runbooks and alerts. Status-page checks, incident templates and on-call docs that name the old provider send the next responder to a page that will be permanently green during your outage.

Frequently Asked Questions

How long does it actually take to migrate off Together AI?

The honest answer is that nobody knows until they have run an evaluation, and the estimates people give before they have are almost always the time it takes to change the client configuration — which is hours. The work that determines the real number is re-establishing output quality on a different model, rebuilding any asset that was derived from Together AI rather than merely stored there, and running both providers side by side long enough to trust the comparison. Teams that have an eval suite and a single construction point for their calls measure this in days. Teams that have neither discover the number by shipping a regression, and then measure it in whatever the incident costs.

What is the difference between exiting Together AI and failing over from it?

Failover is temporary, automatic and reversible: traffic moves because something is broken, and it comes back when the outage ends. Exit is permanent, deliberate and one-directional — you stop paying, the account closes, the derived assets you built inside the vendor stop being available and your obligations around customer data transfer to whoever answers next. A working failover path is genuinely useful evidence that an exit is possible, but it is not an exit plan, because failover deliberately preserves everything that makes exit expensive: the prompts, the schemas, the index, the evaluation baselines and the assumption that you are coming back.

Can we just point the same code at a different provider?

Frequently yes at the transport layer, and that is exactly why exits get underestimated. An identical request shape guarantees that a call succeeds; it guarantees nothing about the answer, the latency distribution, the tokenisation, the way the model handles your system prompt, or whether a structured-output constraint is enforced the same way. The parts of your product that were quietly tuned to one model — prompt wording, retry deadlines, streaming cadence, the number of results you render, how aggressive your validation is — do not move with the base URL. Treat wire compatibility as a convenience during the cutover and never as evidence that the migration is finished.

What do we have to ask Together AI for before the account closes?

Anything that exists only inside their console. In practice that means your historical request and usage records, invoices and any per-request metadata you rely on for cost attribution or incident reconstruction, plus written confirmation of deletion for the data you uploaded and for anything retained in logs. Ask for these while you are still a paying customer with an account manager who answers, not in the week after cancellation. The general rule for exits is that access ends before your need for the records does, so treat the export as a step that happens before the notice, not after it.

Should we run both providers at once during the cutover?

Yes, and you should plan for the double billing rather than being surprised by it. A shadow window where the new provider answers the same requests without those answers reaching users is the only cheap way to compare quality on real traffic, and a ramp by feature — starting with the one where a bad answer is least costly — is safer than a ramp by traffic percentage across a mixed workload, because a percentage split hides which use case regressed. Set the window's end date against the notice date on your existing agreement, not against a feeling that things look fine.

We are not planning to leave. Is any of this worth doing now?

The parts that cost nothing are worth doing today, because every one of them is also an availability control. A single construction point for provider calls, the model identifier and provider recorded on every stored result, an evaluation suite with a current baseline, and your own copy of the request ledger are all things you want during an outage, a price change or a model deprecation — the exit is simply the case that uses all four at once. The expensive parts, like re-embedding a corpus or re-training an adapted model, are worth pricing rather than doing, so that the number exists before somebody has to make the decision under time pressure.

Related Together AI Guides

Migrating? Watch Both Sides of the Wire

During a cutover you are paying two providers and comparing them on live traffic. API Status Check monitors Together AI and its alternatives from outside your infrastructure, so a degradation on either side shows up as an alert instead of as a data point that quietly ruins the comparison.

Start Your Free Trial →

Leaving Together AI: What Actually Holds You In Place