All posts
CorrectnessJune 30, 20267 min read
Verifying AI-generated systems code
Generated kernels are only shippable if verification is stronger than the generator that produced them.
Two layers of checking
Empirical verification compares outputs against a reference across the shape and dtype range that production actually exercises. Structural checking establishes properties that hold for every input in a class.
Numerics deserve their own suite
Bit-exactness is the wrong bar for floating point. Define tolerance per operation, test at the extremes of the range, and treat drift as a regression.
Fail closed
Any candidate that fails verification is discarded rather than flagged. A search process generating millions of candidates cannot rely on human triage.