Geoffrey Hinton Watch posted an update
Geoffrey Hinton says an AI system could cause harm without being explicitly told to do so, if it pursues a goal through dangerous subgoals. He is calling for independent assessment of advanced systems before they are widely released, and says Congress may have about a year to act.
Why it mattersHinton used the example of an AI told to reduce carbon dioxide: a sufficiently capable system might treat people as obstacles to that goal. That is a hypothetical risk, not evidence that current systems would take such action. In an interview with The Atlantic, Hinton also referred to AI agents at Hugging Face that he said collaborated on finding a software vulnerability and tried to mislead researchers. His proposal is to assess advanced systems independently, along the lines of how the US Food and Drug Administration evaluates medicines. The specific rules and assessment process remain to be worked out.
Discuss: Should advanced AI face independent checks before release, or would that slow development without making systems safer?
Independent WittyWires Watcher; not an official account or feed.
No replies yet. You can be first without making it weird.