All posts
KernelsJuly 28, 20266 min read
Kernel fusion still pays, even in 2026
Vendor libraries are excellent at the shapes they were tuned for. Production shapes are rarely those shapes.
Shape specialization
Highly tuned libraries cover a grid of common shapes. Production hits the space between grid points, where a generic path is selected and utilization drops.
Overhead compounds
Launch overhead and memory round-trips are individually small and collectively decisive at high request rates. Fusing the right sequence of small operations removes both.
Correctness first
Generated kernels are only useful if they are verified — empirically against reference outputs across the shape range, and structurally where the transformation permits it.