What is an AI gateway, and do I need one?

An AI gateway is a layer that sits between your application and the large language model providers you call (OpenAI, Anthropic, Google, open-weight models, and so on). Instead of hardcoding one provider SDK, you point your app at the gateway and it handles model routing, authentication to each provider, retries and fallback when a provider is slow or down, cost tracking, rate limiting, and often caching and logging. You need one once you have more than a toy project: the moment you want to switch models without a redeploy, fail over when a provider has an outage, cap spend per customer or team, or see what your app is actually sending and spending. For a single call to a single model, a gateway is overhead; for a product, it is the difference between a demo and something you can operate.

What is the difference between OpenRouter, Portkey, and LiteLLM?

They solve the same problem with different bargains. OpenRouter is a fully hosted router: you get one API key and access to 300+ models across providers, with automatic fallback and routing variants, and you pay a small platform fee on top of provider usage. LiteLLM is open source and self-hostable: you run the proxy yourself, get one OpenAI-compatible API over 100+ providers, set per-team budgets and keys, and pay only for your own infrastructure (a managed cloud tier also exists). Portkey is the enterprise governance option: a gateway plus a hosted control plane for observability, guardrails, semantic caching, prompt versioning, and role-based access. In short, OpenRouter optimises for fastest setup, LiteLLM for control and cost at scale, and Portkey for governance and visibility.

How is OpenRouter priced, and does it mark up models?

At the time of writing, OpenRouter passes provider prices through and charges a platform fee on credit purchases rather than marking up inference per token. Reported figures are around a 5.5% fee when you buy credits, with a small minimum (about $0.80) on small transactions, and many models are billed at the underlying provider's rate. Because it is usage-based with no monthly minimum, it is cheap to start and easy to prototype on. Pricing on all of these platforms changes often, so check OpenRouter's current pricing page and the per-model prices before you commit, especially if you run high volume where even a few percent matters.

Is LiteLLM really free, and what is the catch?

LiteLLM's proxy and SDK are open source and free to self-host, so you pay only for the servers you run it on and for the model usage itself. The catch is operational: you own uptime, scaling, upgrades, and security of the proxy, and you build or wire up your own dashboards unless you use the managed cloud tier. That trade is worth it when you have the infrastructure skills and want the lowest marginal cost at scale, full control over routing logic, and your keys and logs to stay inside your own environment. If you would rather not run infrastructure, LiteLLM Cloud and the hosted options from OpenRouter and Portkey remove that burden for a fee.

Which AI gateway should I build into an AI SaaS template I plan to sell?

Match the gateway to the buyer. For a template aimed at indie developers and fast shipping, OpenRouter is the friendliest default: one key, no infrastructure, and instant access to many models, so the buyer can run it in minutes. For a template aimed at teams who want to self-host and keep data in their own cloud, wire it to LiteLLM so nothing leaves their environment and they control spend per tenant. For an enterprise-focused template where governance, audit logs, and guardrails matter, Portkey is the stronger fit. Whichever you choose, make the provider layer swappable, document how buyers add their own API keys, and ship a working end-to-end call so the integration reads as production-ready rather than a stub.

OpenRouter vs Portkey vs LiteLLM (2026)

One Layer Between Your App and Every Model

If you are shipping anything with an LLM in it in 2026, you quickly hit the same wall. You started with one provider's SDK, then you wanted a cheaper model for one task, a smarter model for another, a fallback when the primary is down, a cap on spend per customer, and a log of what you actually sent. Hardcoding a single provider does none of that.

An AI gateway solves it. It sits between your application and the model providers and gives you one place to route requests, fail over between providers, track and cap cost, rate limit, and often cache and log. We have compared the AI app frameworks that call models and the agent frameworks that orchestrate them. The gateway is the layer beneath both.

Three tools come up in almost every evaluation: OpenRouter, Portkey, and LiteLLM. They solve the same problem with three different bargains.

OpenRouter: a hosted router — one key, 300+ models, automatic fallback, fastest to ship.
LiteLLM: an open-source, self-hostable proxy — one OpenAI-compatible API over 100+ providers, per-team budgets.
Portkey: an enterprise gateway — observability, guardrails, semantic caching, and governance on top of routing.

OpenRouter: The Fastest Way to Many Models

OpenRouter is a managed service. You create one API key, point an OpenAI-compatible client at OpenRouter's base URL, and you can call 300+ models from OpenAI, Anthropic, Google, Meta, Mistral, and many open-weight providers without signing up for each one.

What makes it attractive:

One account, one bill, every model. No separate provider accounts to manage. You top up credits and spend them across models.
Automatic fallback and routing variants. If a provider is slow or returns an error, OpenRouter can route to another. Model suffixes let you ask for behaviour: a throughput-biased variant (:nitro) or a cheapest-available variant (:floor).
Pass-through pricing. At the time of writing it passes provider prices through and takes a small platform fee on credit purchases (reported around 5.5%, with a small minimum on tiny transactions) rather than marking up every token.
No infrastructure. Nothing to run, nothing to scale.

OpenRouter's superpower is time-to-first-call and breadth of models. Its cost: your traffic and keys flow through a third party, model-level controls are lighter than a full governance platform, and at very high volume the platform fee is money you could avoid by self-hosting.

LiteLLM: Open Source, Your Infrastructure, Your Rules

LiteLLM is the open-source option. It ships as a Python SDK and a proxy server you run yourself, exposing one OpenAI-compatible API over 100+ providers. Your app talks to LiteLLM, and LiteLLM translates to whichever provider each model lives on.

What sets it apart:

Self-hosted and MIT-licensed. The proxy and SDK are free. You pay only for the infrastructure you run it on and the model usage itself.
Keys and logs stay in your environment. Nothing has to leave your cloud, which matters for data-residency and compliance stories.
Per-team keys and budgets. Issue virtual keys, set spend limits per team or customer, and track usage — useful for multi-tenant products.
Configurable routing. Load-balance across deployments, set fallbacks, and pin models in a config file you keep in Git.
Broad surface. Beyond chat it covers embeddings, images, audio, batches, rerank, and more through the same interface.

A managed LiteLLM Cloud tier exists for teams who want the same proxy without running it.

LiteLLM's superpower is control and marginal cost at scale. Its cost: you own uptime, scaling, upgrades, and security, and observability is build-it-yourself unless you add the managed tier or pair it with another tool.

Developer reviewing code and metrics on a screen

Portkey: Governance and Observability on Top

Portkey is the enterprise-leaning option. Its gateway is open source, but the value is the hosted control plane around it: deep observability, guardrails, semantic caching, prompt management and versioning, and role-based access control.

What it adds over plain routing:

Observability by default. Every request is logged with cost, latency, and token counts, with traces and dashboards you do not have to build.
Guardrails. Enforce input and output checks — PII, schema, content policy — in the request path.
Semantic caching. Return a cached answer for a semantically similar prompt, which cuts cost and latency on repetitive traffic. It is the one tool of the three that ships this in most comparisons.
Prompt versioning and governance. Manage prompts as versioned artefacts and control who can change what with RBAC.

At the time of writing Portkey offers a free Developer tier (around 10,000 logs a month with short retention) and a paid Production tier (reported near $49 a month for 100,000 logs, 30-day retention, guardrails, unlimited prompt templates, and RBAC), with custom enterprise plans above that.

Portkey's superpower is seeing and governing everything your AI features do. Its cost: the parts you most want — observability, guardrails, longer retention — live in the managed cloud, so a fully self-hosted, fully free setup is not the point of the product.

Head-to-Head Comparison

Dimension	OpenRouter	LiteLLM	Portkey
Hosting	Fully hosted	Open source, self-host or cloud	Open-source gateway plus hosted control plane
Core strength	Many models, one key, fallback	Control, cost at scale, your cloud	Observability, guardrails, governance
Model access	300+ via one account	100+ providers you bring	Routes to providers you configure
Caching	Basic	Build your own	Semantic caching built in
Cost control	Usage-based platform fee	Per-team keys and budgets	Logs, budgets, and RBAC
Best for	Fastest setup	Self-hosting and scale	Enterprise governance

Read it as three bargains, not a scoreboard. The right gateway is the one whose bargain matches your stage and constraints.

How to Choose

Answer three questions.

Do you want to run infrastructure? If no, you are choosing between OpenRouter and Portkey. If yes, LiteLLM gives you the most control and the lowest marginal cost.
What is your biggest need right now? For the widest model access with the least setup, choose OpenRouter. For observability, guardrails, and governance across a team, choose Portkey. For keys and data that never leave your cloud with per-tenant budgets, choose LiteLLM.
How regulated are you? If audit logs, access control, and guardrails are requirements, Portkey is built for that; LiteLLM gets you there if you self-host and add your own logging.

These are not mutually exclusive. A common pattern is LiteLLM self-hosted as the proxy with Portkey or your own stack for observability, or OpenRouter as a fallback provider configured inside LiteLLM. You also still choose the rest of the stack separately — a local model runtime for offline work, a place to keep API keys and secrets, and a cache or rate-limit layer for traffic.

Which One for the Products You Sell

If you build and sell AI SaaS templates or LLM app starters, the gateway you wire in shapes who can buy and run your product:

Indie and fast-ship buyers: OpenRouter is the friendliest default. One key, no infrastructure, a model picker that works in minutes — ideal for an AI chatbot template.
Teams who self-host: LiteLLM keeps keys and data in the buyer's own cloud and controls spend per tenant.
Enterprise buyers: Portkey's observability and guardrails answer the questions a security review will ask.

Whichever you ship, make the provider layer swappable, document clearly how buyers add their own API keys, and include a working end-to-end call rather than a stub. That is what makes an integration read as production-ready and worth paying for.

The Bottom Line

OpenRouter: a hosted router with one key, 300+ models, and automatic fallback. The choice for the fastest path to many models.
LiteLLM: an open-source, self-hostable proxy with one API over 100+ providers and per-team budgets. The choice for control and cost at scale.
Portkey: an enterprise gateway with observability, guardrails, semantic caching, and governance. The choice for teams that must see and control everything.

Ready to turn what you build into income? List your AI SaaS starter or template on CodeCudos, see the wider picture in our best tech stack for web apps in 2026 guide, or learn how to build and sell MCP servers.

OpenRouter vs Portkey vs LiteLLM 2026: AI Gateways & LLM Routing Compared