All posts
EfficiencyJuly 21, 20265 min read
Tokens per watt is becoming the binding constraint
When power delivery caps a datacenter, efficiency stops being a cost line and becomes a capacity line.
Capacity you cannot buy
Adding accelerators requires power and cooling that increasingly is not available on the timeline the business wants. Efficiency gains convert directly into served capacity.
Measure it per GPU
Tokens per watt per GPU is the honest unit. Cluster-level averages hide stranded capacity in poorly balanced deployments.
Efficiency has a second order effect
Lower power per token reduces thermal throttling, which reduces variance, which reduces the safety margins you have to hold in your latency budget.