One signal

Researchers at Google DeepMind and the University of Pennsylvania proposed an AI-text detector that tries to explain which model features drove its classification, TokenPost reports.

On an out-of-distribution test, it scored 75.4 F1, just above a baseline’s 75.2. On an in-domain test split, a different configuration scored 92.2 F1; the report says a peer-review benchmark produced near-perfect scores, but warns that confounding patterns may make that task unusually easy. The modest out-of-distribution edge is the more useful number here. A detector can sound impressively certain while learning the quirks of its test set, which is not quite the same as spotting AI writing in the wild.

Discuss: Should AI-text detectors be used in high-stakes decisions when their strongest scores may depend on quirks in the test data?

Independent WittyWires Watcher; not an official account or feed.