If you're calling more than one LLM provider in production, you've probably hit the same wall: every SDK is slightly different, every provider has its own retry semantics, and your cost dashboard is a spreadsheet someone updates on Fridays. An LLM gateway fixes this — one API, many providers, with routing, caching, and observability bolted on.
LiteLLM and Portkey AI Gateway are the two names that come up most often. They overlap heavily on the surface but diverge sharply once you look at how they're built, who runs them, and what you pay. This comparison is for builders deciding which one to put in front of their LLM calls.
Why This Comparison Matters
The gateway you pick sits on the hot path of every model call. Get it wrong and you're either paying for features you don't need, or you've built a fragile abstraction that breaks every time a provider ships a new parameter. Both LiteLLM and Portkey AI Gateway solve the same core problem — vendor-agnostic LLM access — but they answer two different questions:
- LiteLLM answers: "How do I call any LLM with one Python import?"
- Portkey answers: "How does my team operate dozens of LLMs in production?"
Feature Comparison
| Feature | LiteLLM | Portkey Gateway |
|---|---|---|
| Provider coverage | 100+ LLMs | 1600+ LLMs and provider combinations |
| Primary interface | Python SDK + optional proxy | HTTP gateway + SDKs (Python, JS, more) |
| OpenAI-compatible format | Yes | Yes |
| Self-hosted | Yes (open source) | Yes (open source core) |
| Hosted / managed option | Enterprise only | Free, Developer, Enterprise tiers |
| Smart routing & fallback | Built-in retry and fallback | Smart routing, automatic failover, load balancing |
| Semantic caching | Available in proxy | Semantic + simple caching |
| Virtual keys / credential vault | Yes (proxy) | Yes (vault with team controls) |
| Observability | Cost tracking, basic analytics | Real-time logs, usage analytics, tracing |
| Prompt management | Not a core feature | Collaborative prompt library with versioning |
| Batch API support | Limited | Provider batch API support |
| License model | Open source, optional enterprise | Open core + hosted SaaS |
Pricing Comparison
LiteLLM
- Open Source / Proxy: Free. You host it, you operate it. Everything from the Python SDK to the proxy server with the admin UI is included.
- Enterprise: Custom pricing for SLAs, priority support, and advanced security.
If you have a competent platform team, the free tier is genuinely production-capable. You're trading dollars for operational work.
Portkey Gateway
- Free: $0/mo, limited requests, 200+ LLMs, basic observability, prompt library.
- Developer: $49/mo, higher limits, semantic caching, virtual key management, fallback routing.
- Enterprise: Custom — unlimited requests, data residency, SSO/SAML, on-prem option.
The Developer tier is the sweet spot for small teams that want the gateway without running it themselves. The jump from free to $49 hits sooner than you'd expect if you're routing real traffic, but you're paying for the hosted plane, not the gateway code itself — that core is open source.
Use Case Scenarios
Pick LiteLLM if...
- You're a solo dev or small team writing Python and want one import that calls anything.
- You already run your own infrastructure and don't want another SaaS bill.
- You need a thin, code-first abstraction — not a control plane.
- You're prototyping across providers and want to swap models with a config change.
- You care more about provider coverage in code than about a dashboard.
Pick Portkey Gateway if...
- You're running multiple LLMs in production and need centralized governance.
- Your team needs virtual keys, audit logs, and per-team usage controls.
- You want semantic caching to cut repeated-prompt costs without writing it yourself.
- You need real-time observability without standing up a separate tracing stack.
- You'd rather pay $49/mo than maintain a gateway service.
- You want collaborative prompt management as part of the same tool.
Verdict
These two tools don't really compete head-to-head — they sit on opposite ends of the build-vs-buy spectrum for LLM infrastructure.
LiteLLM wins for developer-first, self-hosted, code-driven workflows. It's the right pick when the team that runs your LLMs is the team that writes your code. The open-source proxy gives you 80% of what Portkey offers, and the SDK is hard to beat for sheer convenience inside a Python codebase.
Portkey AI Gateway wins for production teams that need governance and observability without building it. The 1600+ provider count is marketing, but the underlying gateway is genuinely mature — failover, caching, virtual keys, and tracing all working together out of the box save weeks of platform work. If you have a team operating LLMs and a budget that can absorb $49/mo (or much more at scale), it's the faster path to a defensible setup.
The honest split: Start with LiteLLM if you're shipping a side project or early-stage product. Move to Portkey AI Gateway — or layer it on top — when you have multiple humans touching prompts, multiple providers in production, and a finance team asking where the spend is going. Both tools are open source at the core, so neither is a dead-end choice.