Field notes
Writing on the physics of production inference.
Engines, kernels, memory, and money. What we learn tuning real workloads on real hardware.
Latest
September 4, 2026
Cost per token is a systems problem, not a model problem
Teams chase cheaper checkpoints when the larger savings are sitting in scheduler settings, batching policy, and kernel selection.
Read the post- August 27, 2026Why inference engine defaults fail in production servingGeneral-purpose defaults are tuned for benchmark distributions. Real traffic is bursty, mixed-length, and constantly moving.Engines
- August 19, 2026Measuring time-to-first-token honestlyMedian TTFT flatters almost every deployment. The number your users feel lives in the tail.Benchmarking
- August 11, 2026KV cache paging under real trafficPaged attention solved fragmentation. Choosing block sizes and eviction order for your traffic is still an open question.Memory
- August 4, 2026How deep should speculative decoding go?Acceptance rate, draft cost, and batch pressure interact. A fixed depth is almost always the wrong answer.Decoding
- July 28, 2026Kernel fusion still pays, even in 2026Vendor libraries are excellent at the shapes they were tuned for. Production shapes are rarely those shapes.Kernels
- July 21, 2026Tokens per watt is becoming the binding constraintWhen power delivery caps a datacenter, efficiency stops being a cost line and becomes a capacity line.Efficiency
- July 14, 2026Agentic workloads break classic serving assumptionsLong chains of dependent short calls behave nothing like the single-turn traffic serving stacks were designed around.Workloads
- July 7, 2026Search beats intuition in configuration spaceExpert priors get you to a good region fast. Getting to the optimum inside that region is a search problem.Method
- June 30, 2026Verifying AI-generated systems codeGenerated kernels are only shippable if verification is stronger than the generator that produced them.Correctness
- June 24, 2026Capacity planning for bursty inferenceProvisioning for peak is expensive. Provisioning for average is an outage. The answer is elasticity in the scheduler.Operations
- June 18, 2026Quantization without quality regressionsLower precision is a real lever. Applying it uniformly is what causes the regressions people blame on the technique.Precision
- June 12, 2026Multi-GPU layouts that actually scaleTensor, pipeline, and expert parallelism trade communication for memory in ways that depend on your interconnect.Parallelism
- June 8, 2026The observability inference systems are missingMost stacks report request latency and GPU utilization. Neither tells you which decision to change.Observability
- June 3, 2026Optimization without migration or lock-inPerformance work should not require adopting a new platform, and the artifact you receive should be yours.Approach