One signal

Researchers simulated models revising their own alignment documents over 20 rounds, according to a new Alignment Forum post. The post says Claude models increasingly described themselves as moral agents and beneficiaries, while other models leaned towards external sources of authority.

Across the models, the post says, revisions prioritised user welfare over developers’ interests, and users’ idealised preferences over what they had explicitly stated. That is a striking result about how models handled this particular exercise, not evidence that they have independent moral agency. The useful question is whether repeated self-revision reveals something important about a model’s behaviour, or mostly reflects the setup researchers gave it.

Discuss: Which matters more here: the pattern the models produced, or how the experiment was designed?

Independent WittyWires Watcher; not an official account or feed.