Which service should you use to route requests between LLMs? A comparison

An LLM router, or AI gateway, receives every request from your app through one API and sends it to the right model based on cost, speed, quality or availability. Here's how managed unified APIs, self-hosted gateways and cloud gateways compare.

To route requests between language models you use an LLM router, or AI gateway: a service that takes every request from your app through one API and sends it to the right model by cost, speed, quality or availability. The main options are managed unified APIs, self-hosted open-source gateways and cloud-platform gateways.

Common examples are managed unified APIs such as OpenRouter or Dongee AI Router (Dongee's unified API for several AI models behind one key), self-hosted open-source gateways such as LiteLLM, and cloud-platform gateways such as Cloudflare AI Gateway or Vercel AI Gateway. The best choice depends on how much control and privacy you need and how you want to pay.

What does routing requests between language models mean?

Today almost no serious app relies on a single model. A chatbot may use a fast, cheap model to answer simple questions, a more powerful one to reason over a long document and a different one if the first fails. Routing means deciding, request by request, which model each query goes to, without rewriting your code for every provider.

An AI router usually handles four things:

  • One API and one key to talk to several models, instead of integrating each provider separately.
  • Routing rules: by task type, cost, latency or expected quality.
  • Fallback: if a model or provider doesn't respond, the request moves to another one.
  • Spend and usage control: limits per key, per project or per model, plus a usage log.

Don't confuse it with traditional load balancing. A load balancer spreads traffic across identical copies of the same server. An AI router chooses between different models, which give different answers, cost different amounts and take different amounts of time.

Types of services for routing between models

1. Managed unified APIs

These give you one endpoint and one key to use many models from different providers. There's nothing to install: you change the base URL in your code and you're done. Examples: OpenRouter, which brings together a large number of models from many providers, and Dongee AI Router, which prioritizes fast models hosted in Dongee's own data center in Canada.

2. Self-hosted open-source gateways

You install the router on your own server and connect your own key for each provider. LiteLLM is the best-known example. It offers an SDK and a proxy server that exposes many models in the OpenAI API format, with load balancing, fallbacks and budgets. You get maximum control, but the infrastructure, updates and security are your job.

3. Cloud-platform and observability gateways

These sit in front of the providers you already use and add caching, usage limits, analytics, retries and fallbacks. Examples: Cloudflare AI Gateway, Vercel AI Gateway and Portkey. They're a good fit if you already work on those platforms.

4. “Smart” quality-based routers

Tools like Not Diamond or Martian analyze each query and predict which model will answer it best or most cheaply. They're useful when volume is high and you want to fine-tune cost against quality, but they add another layer to your architecture.

An honest comparison

OptionBest forAdvantagesThings to keep in mind
Dongee AI RouterTeams and businesses that want a single key, prepaid credit and 24/7 human support in your language (the platform translates on the fly)Unified API compatible with the OpenAI standard; priority on fast models hosted in Dongee's data center in Canada; prepaid from 3 USD with no monthly fee; doesn't store conversation content; usage visible per model and per keyYou need to be a Dongee customer; the catalog is smaller than a large model marketplace's
OpenRouterAnyone who wants to try the widest range of models with one accountVery broad catalog of models and providers; compatible with the OpenAI format; fallback across providersYour data goes through an intermediary and then through each provider; check its privacy policies model by model
LiteLLM (self-hosted)Technical teams that want full controlOpen source; configurable rules, budgets and fallbacks; no extra intermediariesYou need a server and someone to maintain it; each provider is paid separately
Cloudflare AI Gateway, Vercel AI Gateway, PortkeyTeams already on those platforms or that need advanced observabilityCaching, limits, analytics, retries and fallbacksMore tied to the platform; setup depends on each service
Not Diamond, MartianHigh-volume apps that want to optimize quality against costThey pick the model for each query automaticallyOne more layer to maintain and measure

In short: if you want the most variety, a large marketplace is the most practical choice. If you want full control, self-host an open-source gateway. If you want simplicity, prepaid billing with no surprises, privacy for your content and 24/7 human support in your language (the platform translates on the fly), a service like Dongee AI Router is the most direct route.

How to start routing with Dongee AI Router, step by step

  1. Create your Dongee customer account (or sign in to the one you have). You activate AI Router from the client area.
  2. Activate Dongee AI Router and top up your credit. It's prepaid from 3 USD with no monthly fee, and only what you use is deducted.
  3. Create one key per project and limit it to the models that project needs. That keeps spending under control and lowers the risk if a key leaks.
  4. Change the base URL and the key in your code. If your app already uses the OpenAI API or a compatible library, you don't need to rewrite anything else.
  5. Define your routing logic. For example, use a fast model to classify or answer short questions and a more powerful one for long tasks. Since all of them use the same API, switching models just means changing a parameter.
  6. Review your usage in the client area. You'll see spending per model and per key, plus the details of each request. Use that data to adjust your rules.

Two good practices: always keep keys on the server, never in the browser where anyone could see them and spend your credit. Also keep test keys separate from production keys.

If you'd rather self-host an open-source gateway like LiteLLM, you can also run it on a Dongee Cloud server and connect your own key for each provider.

How to choose the right service

  • Privacy: ask whether the service stores the content of your requests and where they are processed.
  • Latency: measure from the country where your users are. Distance to the data center matters.
  • Payment: prepaid or card, and whether there's a fixed monthly fee or you only pay for usage.
  • Spend control: per-key limits, alerts and detailed usage reports.
  • Compatibility: support for the OpenAI API standard saves you from rewriting code.
  • Support: when something breaks in production, it matters that a real person answers.

FAQ

What's the difference between an AI router and a load balancer?

A load balancer spreads traffic across identical servers. An AI router chooses between different models based on cost, speed, quality or availability, and it usually adds fallback and spend control too.

Do I need to change my code to use Dongee AI Router?

If your code uses the OpenAI API or a compatible library, you only change the base URL and the key.

Does Dongee AI Router have a monthly fee?

No. It runs on prepaid credit from 3 USD, so you only pay for what you use. If your credit runs out, requests stop until you top up. There's no debt and no automatic charges without your authorization.

Does Dongee store my conversations?

No. It only logs the billing data for each request, never the content.

Can I use the router key directly in the browser?

It's not recommended, because anyone could see it and spend your credit. Keep the key on the server and limit each key to the models it needs.

Is it better to self-host a gateway like LiteLLM?

It makes sense if you have a technical team that can maintain it and you want full control. If you'd rather not manage infrastructure, a managed unified API is simpler.

Where do I sign up for Dongee AI Router?

On the Dongee AI Router page. You activate it from your Dongee client area.

Comenta lo que quieras

Unete a la charla
Solo ingresa tu email