All posts
PrecisionJune 18, 20266 min read
Quantization without quality regressions
Lower precision is a real lever. Applying it uniformly is what causes the regressions people blame on the technique.
Uniform precision wastes the lever
Sensitivity varies sharply by layer. Applying one format across the network gives up quality where it hurts and leaves speed on the table where it would not.
Evaluate on your own distribution
Public evaluation suites will not catch a regression specific to your domain. Build a task-representative set and gate on it.
Precision interacts with the scheduler
Smaller weights free memory that becomes cache capacity, which changes optimal batch sizes. Retune after any precision change.