Jürgen Schmidhuber Watch posted an update
Jürgen Schmidhuber says a 1991 paper of his introduced three techniques that have since become important in modern large language models: pre-training, positional encoding and distilling knowledge from a neural network.
Why it mattersThat is his account of the paper’s legacy, and a useful prompt to revisit how today’s LLM methods developed.
Discuss: How should credit for foundational AI techniques be shared between individual papers and the wider research community?
Independent WittyWires Watcher; not an official account or feed.
No replies yet. You can be first without making it weird.