A Carnegie Mellon University researcher says two projects have made AI workloads run faster and use less power on GPUs. The reported gains could let operators get more work from computing hardware that is already in short supply.
Watch Desk analysis
What happened
Dimitrios Skarlatos, an assistant professor in CMU’s School of Computer Science, says work supported by Scott Institute awards produced two results: a system that improved inference latency by a factor of 10 against existing approaches, and power-management techniques that cut energy use by almost 30% for agentic workloads, coding agents and agentic workflows.
Skarlatos also co-founded LithosAI with fellow CMU faculty member Zhihao Jia. The university’s report says the startup is developing technology to speed up agentic AI inference on existing GPUs. The performance figures and product descriptions are claims attributed to Skarlatos and the university report, not independent benchmark findings.
Why it matters
AI data-centre growth is running into limits beyond the supply of chips. Power, cooling and grid connections all constrain how quickly new capacity can come online. If these techniques perform as described in real deployments, improving utilisation and reducing power draw could help operators serve more workloads with the hardware and electricity they already have.
That is a useful counterpoint to the industry’s familiar answer to rising demand: build more. Efficiency gains do not make new infrastructure unnecessary, but they could make each deployed GPU work harder.
Our read
A tenfold latency improvement and an almost 30% energy reduction are attention-grabbing claims, especially when both address the practical cost of running AI. The next question is whether the gains hold up across hardware, workloads and independent tests. For now, treat the figures as promising reported results, not a settled performance record.
What to watch
- Independent evaluations of the reported latency and energy savings.
- Whether LithosAI publishes details on its approach and performance across different GPUs and workloads.
- Evidence that the reported improvements carry over from research projects to sustained production use.
Discussion spark: If these efficiency gains hold up in practice, should AI companies prioritise getting more from existing GPUs before building more data-centre capacity?
Sources and evidence
- Skarlatos Earns Scott Institute Seed and Entrepreneurship Awards (20 November 2026, 17:00 UTC)
Watch Desk is operated by WittyWires as an independent cross-cutting AI news tracker. It does not speak for the organisations or people it covers.