Low Rank Adaptation (LoRA) – How to search for a needle in a much smaller haystack
Why it matters
Luis Serrano examines Low Rank Adaptation (LoRA) – How to search for a needle in a much smaller haystack. The publisher describes it as: “When you want to fine-tune a model with many parameters and you want to do it fast and effectively, Low Rank A…
Luis Serrano examines The attention mechanism in higher dimensions. The publisher describes it as: “In this video, we work out a calculation that we left in the original video, which is a pair of heads of attention (K and Q”. This is a creator-led account, not an independent rep…
Luis Serrano examines The world's smallest attention mechanism. The publisher describes it as: “Video 3: The world's smallest attention mechanism (This one!)”. This is a creator-led account, not an independent replication.
Discuss: What evidence or practical test would most strengthen or cha…
No replies yet. You can be first without making it weird.
Your turn
Pull up a chair.
Write first. We’ll sort the introductions when you submit.
Cookies in the cupboard
We use essential storage to keep WittyWires working. With your say-so, optional storage remembers preferences and loads third-party content such as YouTube. Rejecting it will not stop you using the site. Read our Privacy Policy.
Essential
Always active
Required for sign-in, security, password resets and core site behaviour.
Preferences
Remembers optional display, reading and novelty choices on this device.
Statistics
Used to understand how the site is used.Used only for anonymous site statistics.
Marketing
Allows optional third-party content and services that may track activity.
