How to Migrate from OpenAI to Together AI: Complete API Migration Guide (2026)
Updated July 2026 · 9 min read · API Status Check
Quick Answer
Together AI serves an OpenAI-compatible endpoint at api.together.xyz/v1. You keep the openai SDK, change base_url, and swap the model string for a fully qualified Hugging Face-style identifier.
📡 Monitor your APIs — know when they go down before your users do
Better Stack checks uptime every 30 seconds with instant Slack, email & SMS alerts. Free tier available.
Affiliate link — we may earn a commission at no extra cost to you
Why Teams Migrate from OpenAI to Together AI
Together AI's draw is breadth: hundreds of open models behind one OpenAI-compatible endpoint, plus dedicated endpoints and fine-tuning. Teams migrate when they want to evaluate or mix many open models without standing up inference infrastructure per model, or when they need a fine-tune they can keep serving through the same API shape.
Migrate selectively. A full replacement is rarely the right call. Keep image generation, audio, and any single-model workload where you do not need breadth — a specialist provider will usually beat a serving layer on that one model on OpenAI, and move the call paths where Together AI is genuinely better. Because you end up running two providers, an independent monitor on both endpoints stops a provider incident from looking like an application bug.
Step 1: Swap the Client
Start with the smallest change that produces a real response, then fix parameters and models from actual errors rather than guessing up front.
# Before — OpenAI
from openai import OpenAI
client = OpenAI(api_key=os.environ["OPENAI_API_KEY"])
resp = client.chat.completions.create(
model="gpt-4o",
messages=[{"role": "user", "content": "hi"}],
)
# After — Together AI (same SDK, new base_url + model)
client = OpenAI(
api_key=os.environ["TOGETHER_AI_API_KEY"],
base_url="https://api.together.xyz/v1",
)
resp = client.chat.completions.create(
model="meta-llama/Llama-3.3-70B-Instruct-Turbo",
messages=[{"role": "user", "content": "hi"}],
)
# Verify the endpoint is actually reachable before debugging your payload:
curl -s -o /dev/null -w "%{http_code}
" https://api.together.xyz/v1/models \n -H "Authorization: Bearer $TOGETHER_AI_API_KEY"Step 2: Map Your Models
There is no exact equivalence between OpenAI models and Together AI models. Treat this as a starting point and validate each swap against your own evaluation set before cutting traffic over.
gpt-4o→meta-llama/Llama-3.3-70B-Instruct-TurboGeneral-purpose swap. Note the fully qualified org/model identifier — bare names like 'llama-3' will 404.
gpt-4o-mini→meta-llama/Meta-Llama-3.1-8B-Instruct-TurboCheap tier for classification and routing.
gpt-4o (code)→Qwen/Qwen2.5-Coder-32B-InstructStrong code model; one of the clearest cost wins over a flagship general model.
text-embedding-3-small→BAAI/bge-base-en-v1.5Embeddings are available, but dimensions differ from OpenAI's — a full corpus re-embed is required.
fine-tuned gpt-4o-mini→your own fine-tuneTogether's fine-tuning output is served through the same OpenAI-compatible endpoint, which is a real advantage over re-plumbing a custom deployment.
Monitor both providers during the cutover
Running OpenAI and Together AI side by side means two dependencies that can fail independently. Better Stack checks both endpoints from multiple regions and alerts you the moment either degrades. Free tier included.
Try Better Stack Free →Step 3: Fix the Parameter Differences
This is where most migrations actually break. The client swap takes minutes; these differences take a day.
Model IDs are namespaced
Together requires the full org/model string, exactly cased. This is the number-one cause of post-migration 404s: a config carried over from OpenAI holds a short model name that does not resolve.
Model quality varies per model, not per provider
Together is a serving layer over many independent open models. Reliability and behavior differ substantially between two models on the same endpoint, so evaluate the specific model, not 'Together' as a whole.
Turbo/quantized variants
Many models are offered in Turbo or quantized flavors at lower price and lower fidelity. Two IDs that look nearly identical can produce noticeably different output quality — pin the exact variant you evaluated.
Dedicated vs serverless
Serverless endpoints share capacity and show cold-start and contention latency. If your migration is latency-motivated, benchmark a dedicated endpoint before concluding the platform is too slow.
Step 4: Know the Errors You Will Hit
These are the failures that show up in the first week after pointing production traffic at Together AI:
"HTTP 404 model not found"Almost always a namespacing mistake — 'Llama-3.3-70B' instead of 'meta-llama/Llama-3.3-70B-Instruct-Turbo'. Copy IDs verbatim from Together's model list."HTTP 429 Too Many Requests"Serverless tiers meter aggressively per model. Back off with jitter and consider a dedicated endpoint for steady production traffic."Cold-start latency spikes"First request to a less-popular serverless model can be seconds slower than steady state. Warm critical models or use dedicated capacity."HTTP 503 on a single model while others work"Together degradation is frequently per-model rather than platform-wide. Check status.together.ai, then try a second model ID before declaring an outage.Step 5: Keep OpenAI Wired as a Fallback
Do not delete the OpenAI path. The most resilient production setup after a migration is a circuit breaker that fails back automatically, so a Together AI incident degrades quality instead of taking the feature down.
# Circuit breaker: Together AI primary, OpenAI fallback
import time
FAILS = {"count": 0, "open_until": 0.0}
THRESHOLD, COOLDOWN = 3, 300 # 3 errors -> skip primary for 5 minutes
def complete(messages):
if time.time() < FAILS["open_until"]:
return call_openai(messages) # breaker open
try:
out = call_together_ai(messages)
FAILS["count"] = 0
return out
except Exception:
FAILS["count"] += 1
if FAILS["count"] >= THRESHOLD:
FAILS["open_until"] = time.time() + COOLDOWN
alert("Together AI breaker opened — serving from OpenAI")
return call_openai(messages)
# Alert on fallback RATE, not just on errors. A migration that silently
# serves 60% of traffic from the fallback is an outage you never noticed.Cutover Checklist
Before you route traffic
- Run your eval set against both providers on identical inputs
- Move model IDs into config so a deprecation is not a deploy
- Load-test against Together AI's real rate limits, not OpenAI's
- Compare p95 latency, not just average, on your hottest path
- Put the migration behind a percentage flag you can roll back
After you route traffic
- Track error rate and latency per provider, separately
- Alert on fallback activation rate — silent failover hides outages
- Subscribe to status.together.ai and keep an independent check running
- Watch cost per successful request, not cost per token
- Re-test your fallback path monthly; an untested fallback is not one
Frequently Asked Questions
Is Together AI OpenAI-compatible?
Yes. api.together.xyz/v1 accepts OpenAI-shaped chat-completions requests, so the official openai SDK works with base_url changed. The one non-obvious requirement is that model must be the fully qualified identifier, such as meta-llama/Llama-3.3-70B-Instruct-Turbo, not a short alias.
Why do my Together AI calls return 404 after migrating?
The model string. OpenAI accepts short names like gpt-4o; Together requires the namespaced org/model form, exactly cased. A config value carried over from OpenAI, or a lowercased identifier, resolves to nothing and returns 404 rather than a helpful validation error.
Do I need to re-embed my vectors to move embeddings to Together AI?
Yes. Open embedding models such as BAAI/bge-base-en-v1.5 have different dimensions and vector spaces than OpenAI's text-embedding-3-*. Old and new vectors cannot coexist in one index, so plan a dual-write and backfill, or keep embeddings on OpenAI and migrate only generation.
Is Together AI reliable enough for production?
Reliability is per model rather than per platform, because Together is a serving layer over many independent open models. A popular Turbo model on dedicated capacity behaves very differently from a niche serverless one. Pin your model, monitor that endpoint specifically, and keep a fallback.
Should I use serverless or dedicated endpoints after migrating?
Serverless is right for evaluation and bursty low-volume traffic. If you migrated for latency, or you have steady production load, dedicated capacity removes cold starts and noisy-neighbor contention — and it is usually the difference between a migration that feels slower than OpenAI and one that feels faster.
Related Guides
Alert Pro
14-day free trialStop checking — get alerted instantly
Next time Together AI goes down, you'll know in under 60 seconds — not when your users start complaining.
- Email alerts for Together AI + 9 more APIs
- $0 due today for trial
- Cancel anytime — $9/mo after trial