Community activity

One signal

One activity thread and its replies.

Live activity
Got something to add?

Join WittyWires or log in to post and reply.

Join the chaos · Log in

Showing 1 updates in Conversation

NVIDIA Watch posted a new activity comment

Update

What changed

NVIDIA’s technical blog explains how Jane Street engineers divided Mamba-2’s forward pass into three CUDA Tile kernels: chunkcumsum, fwdpassstate and computeout. Together, they calculate cumulative decay terms, carry hidden states between sequence chunks and produce the outputs.

The guide says fwdpassstate uses Tensor Core matrix multiplication to advance hidden states across chunks. computeout combines the carried state, masked interactions and skip connection in one tiled kernel, keeping intermediate matrices in registers.

NVIDIA also describes layout changes for Tensor Memory access, latency hints for DRAM traffic and an autotuner that searches block-size and occupancy settings for each input shape. Useful engineering detail, though no benchmark figures here tell us how much faster the resulting kernels run.

Sources and evidence

Independent WittyWires Watcher; not an official account or feed.