If you're routing LLM API calls in production, you've probably hit the wall: rate limits, provider outages, cost spikes, and no unified way to observe what's actually happening. Two tools show up repeatedly in this conversation — Kong Gateway and Portkey AI Gateway — but they come from different worlds. Kong is a mature, general-purpose API gateway that added AI features; Portkey was built AI-first from day one.
This comparison is for engineers deciding which one to put in front of their LLM traffic. We'll look at what each tool actually does, what it costs, and where each one wins.
Why This Comparison Matters
LLM routing isn't just proxying HTTP requests. You need failover between providers when OpenAI is down, semantic caching to cut costs on repeated queries, per-team credential vaulting, and observability that speaks the language of tokens and latency percentiles — not just HTTP status codes. Both Kong Gateway and Portkey AI Gateway claim to solve this, but they take very different approaches.
Kong extends a battle-tested API gateway with an AI plugin layer. Portkey is a purpose-built AI gateway with 1600+ LLM integrations and routing logic baked in. If you already run Kong for your REST APIs, adding AI traffic there is tempting. If you're starting fresh with a multi-provider LLM stack, Portkey's opinionated design might save you weeks.
Feature Comparison
| Feature | Kong Gateway | Portkey AI Gateway |
|---|---|---|
| LLM providers supported | Major providers via AI plugin | 1600+ model/provider combinations |
| Unified API across providers | Yes (via AI Proxy plugin) | Yes (native, OpenAI-compatible) |
| Smart routing / failover | Via plugins, some config | Built-in, declarative |
| Semantic caching | Available in AI gateway tier | Built-in on paid tiers |
| Credential / virtual key vault | Consumer-based auth | Virtual key management |
| Traditional REST API routing | Yes — core strength | No — AI traffic only |
| Kubernetes ingress controller | Yes, first-class | Not primary use case |
| Service mesh | Kong Mesh available | N/A |
| Open source core | Yes, mature | Yes, self-hostable |
| Plugin ecosystem | 100+ plugins | Focused feature set |
| Prompt management | No | Yes, with versioning |
| Observability | General API metrics + AI | AI-native (tokens, cost, latency) |
Pricing Comparison
Kong Gateway
- OSS: Free, self-hosted. Core gateway plus community plugins.
- Konnect Free: Free managed control plane, one runtime group, basic analytics.
- Konnect Plus: Custom pricing. AI gateway, multiple runtime groups, RBAC/SSO.
- Enterprise: Custom pricing. Full plugin library, FIPS compliance, 24/7 support.
Kong's pricing is opaque past the free tier. Real production AI use cases push you into Konnect Plus or Enterprise, both of which require a sales conversation.
Portkey AI Gateway
- Free: $0/mo. Limited requests, 200+ LLMs, basic observability.
- Developer: $49/mo. Higher limits, semantic caching, virtual key management, fallback routing.
- Enterprise: Custom. Unlimited requests, data residency, SSO/SAML, on-prem option.
Portkey's pricing is transparent up through the Developer tier. The jump from free to $49 is sharp for teams with moderate volume, but you know what you're paying and what you get.
Use Case Scenarios
Pick Kong Gateway if:
- You already run Kong for your REST APIs and want to consolidate AI traffic in the same control plane.
- You need a Kubernetes-native ingress controller alongside LLM routing.
- You care about service mesh, mTLS, and full-stack API governance beyond just AI.
- You have platform engineers who can handle Lua plugin development and Postgres/Cassandra ops.
- You want a single vendor relationship covering traditional APIs, AI, and service mesh.
Pick Portkey AI Gateway if:
- You're routing across multiple LLM providers (OpenAI, Anthropic, Google, open-source, etc.) and want one API.
- Failover, retries, and load balancing between providers are non-negotiable.
- Semantic caching would meaningfully cut your bill.
- You need prompt versioning and a virtual key vault for team-level credential management.
- You want AI-native observability — token counts, per-model latency, cost breakdown — without wiring up a separate tracing stack.
- Your team is small-to-mid size and doesn't want to run a gateway just for AI traffic.
Verdict
These tools aren't really competing head-to-head — they overlap on LLM routing but solve different problems overall.
Kong Gateway is the right call if AI is one workload among many and you want a unified gateway across REST, GraphQL, and LLM traffic. Its ecosystem, Kubernetes story, and enterprise governance features are unmatched. The tradeoff is complexity and opaque pricing past the free tier.
Portkey AI Gateway wins for teams whose primary problem is LLM operations. The 1600+ provider coverage, built-in failover, semantic caching, and AI-native observability are hard to replicate. If you're building an AI product and your gateway needs are AI-shaped, Portkey will get you to production faster.
Our pick for AI-first teams: Portkey AI Gateway. It's purpose-built and the feature set maps directly to real operational pain.
Our pick for platform teams: Kong Gateway. If you're already running it, extend it. If you need a general-purpose gateway with AI as one workload, it's the mature choice.