Open-weight AI gives organisations more freedom to run and change models, but it also leaves them with more of the security work. A new Microsoft and NVIDIA paper recommends checking where model files came from, monitoring them after deployment and keeping controls that can be updated as threats change.
Microsoft AI Watch analysis
What happened
The paper, covered by Help Net Security on 19 October, sorts risks into five categories and 30 more specific types. Its central warning is that a model’s published safeguards may not survive modification, repackaging or fine-tuning, and provider-run protections cannot be assumed to follow a model once it is downloaded.
The companies advise organisations to establish a model’s provenance before judging its behaviour. One suggested step is cryptographic signing, which can help confirm that a downloaded file matches the version originally published. If a model arrives without a signature, they say its integrity should be recorded as an open risk. Their wider advice includes monitoring model versions and repositories, and retaining layered controls such as input filtering, output moderation, access restrictions, rate limits and human oversight.
Why it matters
The practical distinction is who supplies the safeguards. Closed-weight models often come with monitoring and guardrails from their provider; with open-weight models, the organisation running them may have to build and maintain more of that protection itself. A model that performs well in a test is not necessarily an intact or trustworthy file.
That is not just a theoretical concern. The article describes research in which a backdoor was hidden in Alibaba’s open-weight Qwen2.5-7B-Instruct model and triggered when it was used with OpenAI’s Codex CLI. The example illustrates a possible supply-chain attack, not evidence that every downloaded model is compromised.
Our read
The useful advice is reassuringly unglamorous: verify the artefact, not just the answers it gives, then keep watching it. Organisations adopting open-weight models should treat file provenance and post-deployment monitoring as part of the job, rather than assuming the original developer’s safety measures will travel with the weights.
What to watch
- Whether Microsoft and NVIDIA publish the planned follow-up guidance on systems built around models.
- Whether model repositories and distributors adopt practical signing and provenance checks.
- How organisations document and respond to integrity risks in unsigned or modified models.
Discussion spark: Should model repositories be expected to provide signed, verifiable releases by default, or is it reasonable to leave that security burden to the organisations downloading the models?
Sources and evidence
- Source update (19 October 2026, 11:04 UTC)
- VectorInst on X: Our social, economic, and political systems patch vulnerabilities at a human pace. Artificial intelligence operates at computer speed. On October 22, security technologist Bruce Schneier will outline why this speed dispa (19 October 2026, 11:04 UTC)
not affiliated with or endorsed by Microsoft