Last year we took on an intelligent customer service project for a cross-border e-commerce company, which required simultaneous integration of GPT-4o, DeepSeek-V3, Qwen, and Doubao. Each of the four models handled a different segment: GPT-4o handled complex English inquiries, DeepSeek took on Chinese long-tail Q&A, Qwen did product knowledge base retrieval, and Doubao served as the fallback for casual chat. It sounded like a clear division of labor, but once we actually got to work, we discovered that just getting the four SDKs running took nearly three weeks.
Authentication methods are not unified — we wrote four sets of signature logic alone
GPT-4o uses Bearer Token plus Organization ID, DeepSeek is compatible with the OpenAI format but has subtle differences in Header field names, Qwen uses AccessKey plus a signature string, and Doubao's authentication uses yet another scheme. Four of our engineers each integrated one vendor, and in the end the codebase contained four copies of authentication logic — changing a single shared parameter meant digging through four files. What was even more troublesome was API Key rotation: the four vendors' consoles each operated independently, and ops manually rotated keys once a month. Miss one and you'd get an alert in the middle of the night.
SSE streaming chunking is inconsistent — frontend rendering goes completely haywire
The differences in streaming output were even greater. GPT-4o and DeepSeek basically follow OpenAI's SSE format, with the data field carrying delta; Qwen in some scenarios packs multiple tokens into a single chunk; and early versions of Doubao mixed the end marker and error events together. The frontend received four kinds of data streams, and the rendering logic turned into four layers of if-else. When the product manager asked to add a "typewriter pause" effect, all four branches had to be changed.
Error codes and usage statistics — the definitions don't match at all
For rate-limit errors, GPT-4o returns 429, DeepSeek returns 429 but with a different body structure, Qwen returns a business code with HTTP 200, and Doubao uses yet another scheme. Our retry strategy was initially written based on HTTP status codes, with the result that retries never triggered on Qwen's side. Usage statistics were even more of a headache: GPT-4o bills by token but calculates prompt and completion separately, Qwen merges input and output, and Doubao has a separate cache-hit field. At the end of the month, finance reconciled accounts, and the CSV files exported from the four backends all had different formats — manually merging them once took half a day.
Simply put, the essential problem of multi-model integration is this: every vendor speaks its own dialect, while your business layer only wants to hear Mandarin.
Convergence solution: protocol translation and usage aggregation at the integration layer
Our approach was to add a thin adaptation layer between the business code and the model SDKs — not aiming for a large, comprehensive gateway, but first solving the three most painful points. First, a unified authentication entry point: the four vendors' Key and signature logic went into a configuration center, and the business side only recognizes a single internal Token. Second, SSE streaming output was unified into the OpenAI format: the adaptation layer is responsible for re-chunking Qwen's and Doubao's chunks, with end events extracted separately. Third, an error code mapping table that categorizes the four vendors' rate-limit, timeout, and content moderation errors into five internal error types, and the retry strategy only recognizes these five.
For usage aggregation, the adaptation layer asynchronously writes a standardized record after each call: model name, input tokens, output tokens, cache hits, latency, error type. The reports finance needs are generated from this table and no longer depend on the vendors' backends.
We later tried unifying base_url onto token8341 and found that it already handles most of the protocol translation and usage aggregation layers. The token8341 Large Model API Aggregation Platform converges the authentication of GPT-4o, DeepSeek, Qwen, and Doubao into a single Key, unifies the SSE format to be OpenAI-compatible, and also maps error codes. We cut nearly 60% of the code from the adaptation layer we had originally written, keeping only the business-specific routing logic.
Which layers should a model gateway handle
In my experience, a model gateway should handle at least four layers: the protocol translation layer, which converts each vendor's dialect into a unified interface; the usage aggregation layer, which records token consumption under a unified definition; the routing layer, which selects models by task type, cost, and latency; and the observability layer, which records the latency and error distribution of each call. The first two layers save manpower; the last two save money. The token8341 Large Model API Aggregation Platform has corresponding capabilities across these four layers — especially usage aggregation and error code mapping, which directly saved us half a day of manual reconciliation every month.
Pitfall-avoidance reminder: don't call each vendor's native SDK directly in your business code, even if you're only integrating two. Once you need to add a third model, the amount of change rises exponentially. In addition, the usage statistics definition must be unified at the integration layer — don't wait until finance reconciliation to discover that the four vendors' backends all define tokens differently.
If you're also doing multi-model integration, I suggest first drawing up a table listing each vendor's authentication method, streaming format, error codes, and billing definitions. If there are more than three differences, it's time to consider convergence at the integration layer.
Author: Li Yunlong
Published: November 9, 2026