ngrok AI Gateway Review

ngrok knows private connectivity. That's the real asset here — routing to self-hosted LLMs without opening inbound ports is the one thing the obvious alternative, a raw multi-provider SDK setup, doesn't solve well.

The pricing is clean: $0.05 per million tokens routed, no subscription, BYOK keeps your existing provider rates. That's easy to defend to a board because it's usage-based and reversible if it doesn't work out.

Two gaps: the $0.05 rate lives on the homepage rather than a dedicated pricing page, and there's no changelog to judge momentum beyond the blog. Scoped keys and automatic failover are real, shipped features — not vaporware claims. But you're routing every request through a new hop; that's a dependency, not just a convenience.

The baseURL swap is the whole pitch: point your OpenAI, Anthropic, or Vercel AI SDK client at gateway.ngrok.ai and you get failover, retries, and scoped keys without deploying anything. That's the right shape for how teams actually run multi-model stacks today — nobody wants to hand-roll retry logic across three provider SDKs. BYOK support means you're not forced into ngrok's margin on inference, only the $0.05/million routing fee.

The real question is where this sits in three years. If your failover rules, fallback chains, and access policies all live in ngrok's config, then migrating off means rebuilding a control plane you didn't have to think about before. Private connectivity to self-hosted models without open inbound ports is a genuine security win, though — that's infra ngrok already knows how to build.

Programmable via Terraform, CLI, and API means it fits GitOps workflows, not just dashboards. No public pricing page or changelog listed is a minor diligence gap for a product touching every inference call you make.

$0.05/million tokens for routing. Inference billed at cost with ngrok keys, or direct from provider with BYOK. No subscriptions, no seats, no tiers to compare. Team routing 500M tokens/month pays $25/month in gateway fees. Year 3 at 2x volume: $600/year in fees. Inference cost dwarfs it either way.

Prepaid credits, drawn down as you route. No auto-renewal clause to hunt for — because there's no term. That's the opposite of the hostage contract you usually sign for infrastructure tooling.

ROI is measurable: observability rolls up tokens, latency, errors per app, per developer, per model. That's the number you'd otherwise piece together from separate provider dashboards. Tradeoff: $1 of starting credit is all you get before a $5 minimum top-up, so real load testing starts on your dime. And BYOK billing lands on two invoices — ngrok's routing fee, provider's inference bill — not one clean line item.

Changing baseURL to gateway.ngrok.ai and dropping in an access key is the whole onboarding story. That's the right instinct — no sidecar, no extra infra, works with OpenAI SDK, Anthropic SDK, and Vercel AI SDK out of the box. The private connectivity to self-hosted models without opening inbound ports is the standout: that's ngrok's actual core competency leaking into a new product, not a bolted-together feature.

What's unproven is the daily-fight surface. Fallback model lists per request and automatic retries sound clean in the docs answer, and the management API is documented — POST to api.ngrok.ai/access-keys — but there's no changelog and no dedicated pricing page, just the rate on the homepage. Scoped keys per app/developer is the kind of access-control feature that matters at week 3, not day 1, and it's here.

$0.05 per million tokens plus pass-through inference cost is easy to model, no subscription lock-in. Tradeoff: new accounts get $1 in credit and then a $5 minimum purchase, so kicking the tires at real volume means buying in.

The pitch is dead simple: point your OpenAI, Anthropic, or Vercel AI SDK client at gateway.ngrok.ai, swap the key, done. No infrastructure to deploy. That's a real ten-minute setup, not a sales-call setup, and the docs presence backs that up.

What sells me is the boring stuff nobody screenshots: automatic retries, failover to a healthy model when one chokes, scoped keys per app instead of one master key everyone shares. That's the stuff that saves you at 2am, not the stuff that wins the demo. Observability rolling up tokens, latency, and errors by app or developer is genuinely missing from most provider dashboards, so that's a fair claim.

$0.05 per million tokens plus inference cost, no subscription, is easy to reason about. The tradeoff: $1 of starting credit, then a $5 minimum purchase, so you're paying to find out if the failover logic behaves the way the docs say under real load.

$0.05 per million tokens is easy to understand. BYOK means you're not trapped on their inference markup, which is the right call and rare in this category. Scoped keys and per-model permissions are real, specific, checkable claims — not vapor.

But the switch cost is baked in structurally. Your baseURL becomes gateway.ngrok.ai. Every SDK call routes through them. If ngrok.ai deprioritizes this (it's an extension of a tunneling company's brand, not their core product), reverting means touching every client config again. That's not catastrophic, just not free.

There's no changelog and no dedicated pricing page, though the documented management API at api.ngrok.ai does back the 'configurable entirely through APIs' claim. The self-hosted LLM routing over private connectivity is a genuine differentiator versus point-to-point setups. Whether ngrok sustains this beyond a launch feature is the open question, not the tech itself.