Langfuse has launched its AI Gateway in public beta, giving organisations a central point to route model requests and track their use. The practical draw is not just another bill counter: teams can see who is using which models, set spend limits per key and, if they opt in, turn requests into searchable traces.
ClickHouse Watch analysis
What happened
The gateway sits between applications and AI providers. In the beta it supports OpenAI and Anthropic; Langfuse says OpenRouter, AWS, GCP, Azure and custom inference APIs are expected to follow. Developers can try it by changing their application’s base URL and using a gateway key.
By default, Langfuse says the gateway records request metadata, usage and cost, not prompt or response content. Organisations can opt into full logging, choose to record nothing, or self-host the open-source gateway on their own infrastructure. Langfuse says self-hosting requires a single 22MB Docker image, and reports median latency below 3ms with tracing enabled.
Langfuse, which joined ClickHouse in January, says its gateway is designed to make AI traffic visible across coding agents, internal assistants and other applications, rather than only tools teams have instrumented individually.
Why it matters
As AI use spreads across departments and software, organisations can lose track of which models are being used, what they cost and what data is being recorded. A shared gateway offers one way to see and govern those requests without requiring each team to build its own reporting layer.
The controls make the privacy trade-off unusually concrete: organisations can collect cost and usage data without logging prompts, or opt into fuller traces for debugging and evaluation. The beta supports only two providers so far, and Langfuse’s performance figures are its own, but the setup gives teams a practical way to test central oversight.
Our read
This is a useful answer to a real operational headache: AI has escaped the single-chatbot era, and finance and platform teams need more than a pile of API invoices. The option to leave content logging off by default is a sensible starting point. The test will be whether teams can adopt the gateway without turning every model call into a new piece of plumbing.
What to watch
- When Langfuse adds the other providers it says are coming.
- Whether teams can enforce gateway use across varied applications and coding agents.
- How the gateway performs under real-world loads, beyond the company’s published figures. For the Activity teaser: Langfuse’s new AI Gateway puts model traffic, usage and costs behind one entry point, with optional tracing and spend limits. The public beta supports OpenAI and Anthropic; the open-source gateway can also be self-hosted. That could help teams see where AI is being used without logging prompt content by default. Our full story looks at the controls, privacy choices and what the beta still needs to prove.
Discussion spark: Should companies route all AI requests through a shared gateway for oversight, or would that add too much friction to teams building with models?
Sources and evidence
- Source update (9 November 2026, 16:57 UTC)
Independent WittyWires tracker for public updates about ClickHouse. Not affiliated with or endorsed by ClickHouse; this is not an official account.