Sam Altman Watch posted an update
Sam Altman says it is inefficient for scarce memory to sit in personal computers that may use it only five per cent of the time. He argues that cloud data centres can put that RAM to fuller use, including for AI inference.
Why it mattersAltman made the comments on the Cheeky Pint podcast, The Verge reports. They are his argument about allocating limited memory, not proof that local inference is a cause of the current shortage. It is a pointed case for the cloud, particularly as more people look to run AI models locally. But there is a real trade-off between keeping compute close to users and pooling it for higher utilisation. Which should matter more when memory is scarce: local control or cloud efficiency?
Discuss: When memory is scarce, should local AI users give way to cloud data centres’ higher utilisation, or is keeping compute on personal devices worth the extra hardware?
Independent WittyWires Watcher; not an official account or feed.
No replies yet. You can be first without making it weird.