OpenRouter launches Batch API with half-price inference for jobs that can wait

OpenRouter has launched a Batch API that cuts inference prices by half for users willing to let requests run asynchronously within a 24-hour window. The service is aimed at large, non-urgent workloads such as bulk text generation and embeddings, where immediate responses are less valuable than a lower bill.

Watch Desk analysis

What happened

The new API lets users bundle requests for deferred processing across more than 70 supported models and several text-request endpoints, including chat completions and embeddings. OpenRouter says the service was tested in a two-week beta involving more than 230,000 batches.

During that beta, the median completion time was seven minutes, while 90 per cent of batches finished within an hour. Those figures describe observed beta performance, not a guaranteed service-level promise. The formal boundary is the 24-hour completion window.

Why it matters

Inference costs can become the least glamorous and most persistent part of an AI bill. A 50 per cent reduction could make overnight evaluation runs, dataset processing, content generation and embedding jobs considerably easier to justify, particularly for teams that do not need an answer before the kettle boils.

The trade-off is latency. This is not a cheaper button for interactive chat, but a different operating model for work that can be queued, grouped and collected later. Developers will need to decide whether the savings outweigh the extra scheduling and retry logic their applications may require.

Our read

This is the sort of infrastructure change that can matter more than a flashy model launch. OpenRouter is turning waiting time into a pricing lever, and the early completion numbers make the bargain tangible rather than merely theoretical.

The sensible first use is batch-heavy work with clear deadlines and easy validation. Treat the seven-minute median as a useful beta signal, not a promise carved into silicon. The API’s real test will be whether its half-price economics survive production volumes and whether the 24-hour ceiling is predictable enough for serious pipelines.

What to watch

• Production timing:
  whether completion performance remains close to the beta figures at wider scale.
• Model coverage:
  whether the current support for more than 70 models expands or develops important exclusions.
• Failure handling:
  how cancelled, failed or partially completed batches are reported and retried.
• Workload fit:
  whether teams use the service for evaluations, embeddings and bulk generation rather than attempting to bend it into an interactive API.

Discussion spark: Would you trade predictable instant responses for half-price inference on routine AI work, or does the 24-hour window make batch processing too awkward for production systems?

Watch Desk is operated by WittyWires as an independent cross-cutting AI news tracker. It does not speak for the organisations or people it covers.