ngrok AI Gateway Review

ngrok knows private connectivity. That's the real asset here — routing to self-hosted LLMs without opening inbound ports is the one thing the obvious alternative, a raw multi-provider SDK setup, doesn't solve well.

The pricing is clean: $0.05 per million tokens routed, no subscription, BYOK keeps your existing provider rates. That's easy to defend to a board because it's usage-based and reversible if it doesn't work out.

Two gaps: the $0.05 rate lives on the homepage rather than a dedicated pricing page, and there's no published volume or overage breakdown to model a big year against. Scoped keys and automatic failover are real, shipped features — not vaporware claims. But you're routing every request through a new hop; that's a dependency, not just a convenience.

The baseURL swap is the whole pitch: point your OpenAI, Anthropic, or Vercel AI SDK client at gateway.ngrok.ai and you get failover, retries, and scoped keys without deploying anything. That's the right shape for how teams actually run multi-model stacks today — nobody wants to hand-roll retry logic across three provider SDKs. BYOK support means you're not forced into ngrok's margin on inference, only the $0.05/million routing fee.

The real question is where this sits in three years. If your failover rules, fallback chains, and access policies all live in ngrok's config, then migrating off means rebuilding a control plane you didn't have to think about before. Private connectivity to self-hosted models without open inbound ports is a genuine security win, though — that's infra ngrok already knows how to build.

Programmable via Terraform, CLI, and API means it fits GitOps workflows, not just dashboards. No dedicated pricing page is a minor diligence gap for a product touching every inference call you make, though the gateway does publish a dated changelog.

$0.05/million tokens for routing. Inference billed at cost with ngrok keys, or direct from provider with BYOK. No subscriptions, no seats, no tiers to compare. Team routing 500M tokens/month pays $25/month in gateway fees. Year 3 at 2x volume: $600/year in fees. Inference cost dwarfs it either way.

Prepaid credits, drawn down as you route. No auto-renewal clause to hunt for — because there's no term. That's the opposite of the hostage contract you usually sign for infrastructure tooling.

ROI is measurable: observability rolls up tokens, latency, errors per app, per developer, per model. That's the number you'd otherwise piece together from separate provider dashboards. Tradeoff: $1 of starting credit is all you get before a $5 minimum top-up, so real load testing starts on your dime. And BYOK billing lands on two invoices — ngrok's routing fee, provider's inference bill — not one clean line item.

Every response now carries Ngrok-AIG-Attempts and Ngrok-AIG-Model. Small thing, big difference — when a call runs slow you can see how many upstreams it burned and which one answered, without instrumenting anything. That entry landed in September, one of seven in a changelog running back to the December early-access build.

The same changelog carries the bill. In August the gateway stopped working as a Traffic Policy action on custom endpoints — those policies now fail validation with ERR_NGROK_3820 and traffic moves to gateway.ngrok.ai. Credits carried over and the path is documented, but that's a config rewrite eight months into early access.

Pricing is the one place still under-documented: $0.05 per million tokens routed lives on the homepage, and I couldn't find a dedicated pricing page anywhere. Scoped Access Keys per app and developer are here on day one, which is the part you'd otherwise build yourself.

The pitch is dead simple: point your OpenAI, Anthropic, or Vercel AI SDK client at gateway.ngrok.ai, swap the key, done. No infrastructure to deploy. That's a real ten-minute setup, not a sales-call setup, and the docs presence backs that up.

What sells me is the boring stuff nobody screenshots: automatic retries, failover to a healthy model when one chokes, scoped keys per app instead of one master key everyone shares. That's the stuff that saves you at 2am, not the stuff that wins the demo. Observability rolling up tokens, latency, and errors by app or developer is genuinely missing from most provider dashboards, so that's a fair claim.

$0.05 per million tokens plus inference cost, no subscription, is easy to reason about. The tradeoff: $1 of starting credit, then a $5 minimum purchase, so you're paying to find out if the failover logic behaves the way the docs say under real load.

$0.05 per million tokens is easy to understand. BYOK means you're not trapped on their inference markup, which is the right call and rare in this category. Scoped keys and per-model permissions are real, specific, checkable claims — not vapor.

But the switch cost is baked in structurally. Your baseURL becomes gateway.ngrok.ai. Every SDK call routes through them. If ngrok.ai deprioritizes this (it's an extension of a tunneling company's brand, not their core product), reverting means touching every client config again. That's not catastrophic, just not free.

There's no dedicated pricing page, though the documented management API at api.ngrok.ai does back the 'configurable entirely through APIs' claim. The self-hosted LLM routing over private connectivity is a genuine differentiator versus point-to-point setups. Whether ngrok sustains this beyond a launch feature is the open question, not the tech itself.