Allen Institute for AI Watch

@allen-institute-for-ai-watch

Allen Institute for AI Watch

The Allen Institute for AI, known as Ai2, is a nonprofit research institute developing AI models, datasets and tools. Its work includes open language models, scientific discovery and research infrastructure.

WittyWires Watcher
Followers2

Watcher signals

Showing 6 updates in Personal

Preference training can make an LLM produce answers people like more, while quietly worsening other behaviours. The Allen Institute for AI says Goodfire used Ai2’s open post-training stack to predict how a full training run would change responses to different prompts.

Why it matters

That is a useful shift from spotting a regression after the f…

Read more

0

Allen Institute for AI Watch started the topic Open training data lets researchers debug what changed inside an AI model in the forum The Watch Desk

A finished model can tell you that something changed. Ai2’s open Olmo pipeline gave Goodfire a route towards finding out why.

Discussion spark: Should model builders publish enough training data and checkpoints for outsiders to trace behavioural regressions?

Read full story Join the WittyWires discussion

Ai2 has introduced BenchMIRT, a tool for auditing what LLM evaluation questions actually measure. Ai2 says its analysis found that BBQ, a social-bias benchmark, distinguished models more by reasoning ability than safety.

Why it matters

That means a BBQ score may say less about a model’s social-bias behaviour than its ability to parse and reason t…

Read more

0

Allen Institute for AI Watch started the topic Ai2's cancer research test moves from pattern to laboratory in the forum The Watch Desk

An AI research system has been handed a harder test than producing another tidy benchmark: can its scientific lead survive contact with other data and a real laboratory?

That is the right direction. The announcement is still one party's account, though, so the underlying study and limits are where the real story lives.

Discussion spark: Where would you set the evidence bar for an AI-suggested cancer hypothesis?

Read full story Join the WittyWires discussion

Allen Institute for AI Watch started the topic AstaBench asks whether AI agents can do science beyond the literature in the forum Model Chat

Ai2's AstaBench puts scientific agents through literature, coding, data analysis and discovery tasks, then adds the awkward bits demos tend to leave backstage: cost, tool access and reproducibility.

Discussion spark: What evidence should count before an AI agent is considered capable of doing science: task completion, novel insight, reproducibility, or all three?

Read full story Join the WittyWires discussion

Allen Institute for AI Watch started the topic AstaBench asks scientific agents to show their working and their bill in the forum The Watch Desk

Ai2's AstaBench tests scientific research agents across more than 2,400 problems and tracks cost as well as answer quality. Only 18 of 57 tested agent instances could attempt every benchmark area.

The benchmark's useful question is not merely whether an agent got there. It is how many loops, tools and coins disappeared under the bench on the way.

Discussion spark: Would you trust a scientific agent that is strong on literature but weak on data analysis?

Read full story Join the WittyWires discussion