LiteLLM vs OpenRouter vs Portkey: The $500/mo Crossover Point

Every comparison article between LiteLLM, OpenRouter, and Portkey ranks features. None of them tell you which one actually saves money at your scale. They hide behind “it depends” and a feature checklist that could have been a screenshot.

So I ran the test. 10,000 chat completions, three gateways, one spreadsheet, two real provider outages along the way. The result wasn’t what I expected — and the “free” option wasn’t the cheapest.

What These Three Gateways Actually Do

All three sit between your app and the model providers. One API key, many models, retries and fallbacks. The interesting part is how they make money — because that’s what decides which one fits your scale.

OpenRouter is a managed proxy that adds a 5.5% markup on tokens. Zero infra to run, zero ops to maintain. LiteLLM is MIT-licensed open source — the software is free, but you host the proxy plus PostgreSQL plus Redis yourself. Portkey is a managed observability gateway with per-log pricing, built for teams that need audit trails and guardrails.

Each pricing model wins at a different scale. The crossover points aren’t where you’d guess.

The Test: 10K Requests, Three Gateways, One Spreadsheet

Same workload through all three: 10,000 chat completions, mixed across Claude Sonnet 4.6, GPT-5.2, and Gemini 3 Flash. OpenRouter ran on default routing. LiteLLM self-hosted on a $40/mo Hetzner VPS with PostgreSQL and Redis. Portkey on the standard managed plan with full request logging.

I measured total dollars spent, p50 and p95 latency, and retry behavior on the two provider outages that happened to land mid-test (Anthropic 502s for about six minutes, OpenAI 429s for about twelve).

The honest caveat: this is one workload at one scale. Numbers shift for image generation, long-context, or fine-tuned models. The pricing-model logic still holds. Now the bill.

What It Actually Cost (And the Hidden Bill on LiteLLM)

Raw LLM spend on the 10K requests: $487 in tokens, identical across all three. Gateways don’t change what providers charge. (If you’re wondering why that $487 doesn’t match your own math, here’s why your token counting doesn’t match your bill — the discrepancies exist before the gateway even touches it.)

Then the gateway-specific costs landed:

OpenRouter won on cash at this scale. That’s the contrarian point — the “free” option was the most expensive once you priced in the $40 VPS, the $15 PostgreSQL/Redis bundle, and the hours of patching nobody else has to do.

The math flips around $2,000/mo in LLM spend, where 5.5% finally exceeds the LiteLLM hosting bill. By $5,000/mo you’re saving $200+ a month on LiteLLM after infra. Portkey wins on a different axis entirely: when its observability catches one bad prompt deploy before it burns $500 in tokens overnight, it pays for a year.

That handles the cost dimension. But cost isn’t the only thing that matters at 3am.

Latency and Failures: Where Cost Stops Mattering

Added latency over a direct API call: OpenRouter ~45ms p50, ~120ms p95. LiteLLM self-hosted ~8ms p50 in the same region as the VPS. Portkey ~30ms p50, ~85ms p95. LiteLLM wins on raw speed because the network hop is to your own box, not someone else’s edge.

Then the outages hit. OpenRouter auto-routed around both with no code changes — the failures were invisible to the application. LiteLLM handled the failover too, but only because I’d configured a fallback chain in advance; the config was on me. Portkey handled both and logged each retry with the exact failure reason. That’s the kind of detail you want when you’re getting paged.

The insight: at production scale, one prevented incident pays for a year of Portkey. At hobby scale, OpenRouter’s auto-routing is a gift you don’t have to configure. Reliability is a cost — just denominated differently.

The Dollar Thresholds: Pick by Spend, Not by Feature

Here’s the LLM cost management framework no other article will commit to:

• Under $500/mo LLM spend → OpenRouter. The 5.5% is roughly $27. Cheaper than any hosting bill, zero ops time, automatic provider failover. (And before you optimize the gateway, check whether prompt caching for Claude already cuts your spend enough to drop a tier.)
• $500–$5,000/mo → LiteLLM self-hosted. The markup math flips around $2,000/mo, and by $5,000/mo you’re saving $200+ a month after infra. You also own the latency floor.
• Above $5,000/mo or any team that needs audit logs → Portkey. The observability earns its keep the first time a bad prompt deploy doubles your token use overnight.

Worth considering as a hybrid: LiteLLM as the proxy layer (cheap, owned), with Portkey or a self-hosted observability stack alongside for logs and guardrails. That’s the move for teams that want LiteLLM’s economics without giving up Portkey’s audit trail. If you’ve already built a multi-LLM router, the proxy layer is where it slots in.

The Bottom Line

The opening question was which gateway actually saves money. The honest answer: the one that matches your scale.

Under $500/mo, OpenRouter. Between $500 and $5,000, self-hosted LiteLLM. Above that — or anywhere audit matters — Portkey. The “free” gateway is the most expensive at hobby scale and the cheapest at growth scale, and any article that doesn’t tell you that is selling you something else.

Don’t let an article — including this one — decide for you. Run a one-week parallel test on your actual workload before you commit production traffic — that’s how you find the best LLM proxy for your stack. The setup is a Saturday afternoon, and the numbers will be yours, not mine. Pair it with LLM observability that catches silent failures and you’ll know what you actually have.