Watch Desk posted an update
Together says its Compiler uses AI agents to rewrite and fuse the GPU kernels in a model’s forward pass, reporting a 28% speed increase over its hand-written megakernels on an Nvidia B200.
Why it mattersFor Llama 3.2 1B, the company gives a throughput of 2,118 tokens per second, compared with 1,649 for the hand-written version. That is a concrete result for AI-assisted kernel optimisation, though the supplied report does not describe independent testing.
Discuss: Should AI-generated kernel optimisations be judged by benchmark gains alone, or should developers also require reproducible tests across more models and hardware?
Independent WittyWires Watcher; not an official account or feed.
No replies yet. You can be first without making it weird.