TokenWorks
AIAPIOpenAI

How to Switch Between LLM Providers Without Changing Your Code

TokenWorks Team·2026-09-03

Most people don't start a project thinking about provider portability. They grab a key from whichever vendor is easiest to sign up for, write the integration, and ship. A few months later they read a benchmark, or open a bill, or slam into a rate limit, and they want to try something else. That's when the pain shows up, because they baked assumptions about the first provider into the HTTP layer.

The fix is not some abstraction library. It's a wire format. The OpenAI chat completions API has quietly become the default protocol for talking to LLMs, and once you write against it, switching providers is mostly a matter of changing two strings.

The API that everyone copied

OpenAI defined a simple contract: you POST JSON to /v1/chat/completions, you get back a choices[0].message.content string. The request looks like this:

curl https://api.openai.com/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $OPENAI_API_KEY" \
  -d '{
    "model": "gpt-4o",
    "messages": [{"role": "user", "content": "Explain DNS in one paragraph"}]
  }'

Almost every major model now speaks this dialect. DeepSeek, Qwen, ERNIE, Doubao, and the others all expose an endpoint that accepts the same body and returns the same shape. That means your client code does not care who is on the other end. You can swap the model string and the endpoint and nothing else changes.

Here is the same call pointed at an aggregator that carries several of these models under one base URL:

curl https://api.token8341.com/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $TOKENWORKS_API_KEY" \
  -d '{
    "model": "deepseek-chat",
    "messages": [{"role": "user", "content": "Explain DNS in one paragraph"}]
  }'

One client, many models

In Python you normally instantiate the OpenAI SDK once and pass a model name per request. If your provider supports the compatible API, you set base_url once and keep everything else identical:

from openai import OpenAI

client = OpenAI(
    base_url="https://api.token8341.com/v1",
    api_key="sk-your-tokenworks-key",
)

def ask(model: str, prompt: str) -> str:
    resp = client.chat.completions.create(
        model=model,
        messages=[{"role": "user", "content": prompt}],
    )
    return resp.choices[0].message.content

# Same function, three different model families.
print(ask("gpt-4o", "Summarize this log for me."))
print(ask("deepseek-chat", "Summarize this log for me."))
print(ask("qwen-max", "Summarize this log for me."))

The function does not change. The model string changes. That is the whole trick. Streaming works the same way:

stream = client.chat.completions.create(
    model="deepseek-chat",
    messages=[{"role": "user", "content": "Write a haiku about databases"}],
    stream=True,
)
for chunk in stream:
    delta = chunk.choices[0].delta.content or ""
    print(delta, end="")

Function calling, JSON mode, and embeddings all ride the same compatible surface, so you are not limited to plain chat when you switch.

What switching does not buy you

I want to be clear about the part that does not change for free. The transport is portable; the models are not.

Different models respond to prompts differently. A system prompt tuned for GPT-4o can underperform on a reasoning model that wants a different structure. Context windows differ, sometimes by a lot: one model might take 128k tokens while another takes 32k. Tokenizers differ too, so the same document costs a different number of tokens on each model. Max output length is another knob that is per-model, not per-protocol.

So "switch without changing code" is really "switch without rewriting your HTTP client." You still owe yourself an evaluation run before you point production traffic at a new model. The good news is that the compatible API makes that evaluation cheap to run, because the harness is just a loop over model names.

A practical pattern

What I actually do in side projects is keep a small config table, not code changes:

MODELS = {
    "fast": "deepseek-chat",
    "smart": "qwen-max",
    "frontier": "gpt-4o",
}

def run(task, prompt):
    return ask(MODELS[task], prompt)

Want to try a cheaper model for the fast lane? Change one line. Want to fall back to a second provider when the first is rate limited? Catch the exception and retry with the next model string. None of this touches the request construction.

That fallback pattern is where a multi-model endpoint earns its keep. With one base URL and one key you can route, retry, and A/B test across providers, all through the same two lines of client setup you wrote on day one.

Response quirks to watch for

The transport is compatible, but a few response-level differences still leak through. Reasoning models sometimes return an extra field like reasoning_content alongside the normal message. Code that reads choices[0].message.content keeps working regardless, but you may want to surface that reasoning text in a debug view. Error messages also vary: one provider returns a helpful insufficient_quota string, another returns a bare 429 with no body. If you are building retries, treat any 429 or 5xx as "back off and try again" rather than parsing vendor-specific text.

None of this is a reason to stay locked to one provider. It is a reason to keep your error handling generic and your model choice a variable.

This is the position TokenWorks takes: an OpenAI-compatible endpoint at https://api.token8341.com/v1 where the model string is the only thing that changes when you move between GPT-4o, Claude, Gemini, DeepSeek, Qwen, ERNIE, Doubao, Spark, and Pangu.

← Back to Blog