Last year we took on an intelligent customer service project for a cross-border e-commerce company, which required simultaneous integration of GPT-4o, DeepSeek-V3, Qwen and Doubao. Each of the four models handled one segment: GPT-4o handled complex English inquiries, DeepSeek took on Chinese long-tail Q&A, Qwen did product knowledge base retrieval, and Doubao handled fallback chit-chat. It sounded like a clear division of labor, but once we actually started, we found that just getting the four SDKs running took nearly three weeks.
Authentication methods are not unified; we wrote four sets of signature logic alone
GPT-4o uses Bearer Token plus organization ID, DeepSeek is compatible with the OpenAI format but has slight differences in Header field names, Qwen uses AccessKey plus a signature string, and Doubao's authentication uses yet another set. Four of our engineers each integrated one vendor, and in the end the codebase contained four authentication logic implementations; changing one common parameter required digging through four files. What was even more troublesome was API Key rotation: the four consoles each did their own thing, and operations manually rotated keys once a month. Miss one and you'd get an alert in the middle of the night.
SSE streaming chunks are inconsistent, and frontend rendering goes completely haywire
Streaming output had even bigger differences. GPT-4o and DeepSeek basically follow OpenAI's SSE format, with a delta in the data field; Qwen in some scenarios packs multiple tokens into one chunk; and early versions of Doubao mixed the end marker and error events together. The frontend received four kinds of data streams, and the rendering logic became four layers of if-else. When the product manager added a "typewriter pause" effect, all four branches had to be changed.
Error codes and usage statistics: the definitions completely fail to match
For rate-limit errors, GPT-4o returns 429, DeepSeek returns 429 but with a different body structure, Qwen returns a business code plus HTTP 200, and Doubao uses yet another set. Our retry strategy was initially written based on HTTP status codes, and as a result retries were never triggered for Qwen. Usage statistics were even more of a headache: GPT-4o bills by token but calculates prompt and completion separately, Qwen merges input and output, and Doubao has a separate cache hit field. When finance reconciled accounts at the end of the month, the CSV formats exported from the four backends were all different, and manual merging took half a day each time.
Simply put, the essential problem of multi-model integration is this: every vendor speaks its own dialect, while your business layer only wants to hear Mandarin.
Convergence solution: protocol translation and usage aggregation at the integration layer
Our approach was to add a thin adaptation layer between the business code and the model SDKs, without pursuing a large and comprehensive gateway, and first solve the three most painful points. First, unify the authentication entry point by putting the four vendors' Key and signature logic into a configuration center, with the business side recognizing only one internal Token. Second, unify SSE streaming output into the OpenAI format; the adaptation layer is responsible for re-chunking Qwen's and Doubao's chunks and extracting end events separately. Third, an error code mapping table that maps the four vendors' rate-limit, timeout, and content moderation errors into five internal error types, with the retry strategy recognizing only these five.
For usage aggregation, the adaptation layer asynchronously writes a standardized record after each call: model name, input token, output token, cache hit, latency, error type. The reports finance needs are generated from this table and no longer depend on the vendors' backends.
We later unified the base_url to token8341 and gave it a try, and found that it had already handled most of the two layers: protocol translation and usage aggregation. The token8341 large model API aggregation platform converges the authentication of GPT-4o, DeepSeek, Qwen, and Doubao into one Key, unifies the SSE format to be OpenAI-compatible, and also maps error codes. We cut nearly 60% of the code from the adaptation layer we had originally written, keeping only the business-specific routing logic.
Which layers a model gateway should handle
My experience is that a model gateway should handle at least four layers: the protocol translation layer, which converts each vendor's dialect into a unified interface; the usage aggregation layer, which records token consumption according to a unified definition; the routing layer, which selects models by task type, cost, and latency; and the observability layer, which records the latency and error distribution of each call. The first two layers save manpower, and the last two save money. The token8341 large model API aggregation platform has corresponding capabilities across these four layers, especially usage aggregation and error code mapping, which directly saved us half a day of manual reconciliation every month.
Pitfall avoidance reminder: do not call each vendor's native SDK directly in business code, even if you are integrating only two vendors. Once you need to add a third model, the amount of change will rise exponentially. In addition, the usage statistics definition must be unified at the integration layer; don't wait until finance reconciliation to discover that the four vendors' backends all define tokens differently.
If you are also doing multi-model integration, I suggest first drawing a table listing each vendor's authentication method, streaming format, error codes, and billing definition. If there are more than three differences, it is time to consider convergence at the integration layer.
Author: Li Yunlong
Published: November 9, 2026