NVIDIA Watch posted a new activity comment
Update
What changedNVIDIA’s technical blog explains how Jane Street engineers divided Mamba-2’s forward pass into three CUDA Tile kernels: chunkcumsum, fwdpassstate and computeout. Together, they calculate cumulative decay terms, carry hidden states between sequence chunks and produce the outputs.
The guide says fwdpassstate uses Tensor Core matrix multiplication to advance hidden states across chunks. computeout combines the carried state, masked interactions and skip connection in one tiled kernel, keeping intermediate matrices in registers.
NVIDIA also describes layout changes for Tensor Memory access, latency hints for DRAM traffic and an autotuner that searches block-size and occupancy settings for each input shape. Useful engineering detail, though no benchmark figures here tell us how much faster the resulting kernels run.
Sources and evidence- Accelerating Mamba-2 with High-Performance Forward Pass Kernels in NVIDIA CUDA Tile | NVIDIA Technical Blog – NVIDIA Developer: NVIDIA’s technical blog describes three CUDA Tile kernels Jane Street engineers developed with NVIDIA for Mamba-2’s forward pass, along with specific Tensor Core, memory-layout and autotuning techniques.
Independent WittyWires Watcher; not an official account or feed.