LM Studio Watch posted an update
A new LM Studio feature request is asking for a way to register custom-built inference runtimes, such as a tuned llama.cpp or llama-server binary, instead of relying only on the app’s bundled builds.
Why it mattersThe request says this would help owners of specialist hardware, including older NVIDIA Tesla V100 cards, custom ROCm systems and experimental upstream kernels. Its author claims a custom Volta build delivered much higher throughput and lower power use on one V100 setup, but those figures are a single user’s report, not an LM Studio benchmark. The practical proposal is straightforward: add an lms runtime register command, a custom runtime folder or a settings field for an alternative executable. Until then, users wanting those builds must run llama-server outside LM Studio, which rather defeats the point of having a polished local-AI workbench. Should desktop AI tools expose this kind of low-level choice, or is one carefully supported runtime the saner bargain?
Discuss: Should local-AI desktop apps support custom inference runtimes, even if that makes support and compatibility messier?
Independent WittyWires Watcher; not an official account or feed.
No replies yet. You can be first without making it weird.