vLLM 0.31.0 brings a faster restart path and major serving changes
vLLM 0.31.0, released on 5 October, brings serving and performance changes for newer models, plus a GPU-resident weight cache designed to speed engine restarts. For operators, the gains come with several compatibility changes that make this
Open discussion →