Precision & Quantization
Floats, quantization, and FP8 training that does not blow up.
3 logs in this box-
Loss scaling, FP8, and the failure that never crashes
Why FP16 gradients underflow to exactly zero without a fix, the loss-scaling control loop that recovers them, and how DeepSeek-V3 actually trains in FP8 once naive per-tensor scaling and a 14-bit accumulator both get in the way.
-
Deploying a quantized LLM: GPTQ, AWQ, speculative decoding, and where it breaks
How post-training quantization rounds weights by measuring their actual effect instead of guessing, why speculative decoding is nearly free on memory-bound hardware, and the three specific ways production quantization fails.
-
What a number actually is: two's complement, floating point, and the price of quantization
Why computers count in binary, how negative numbers and fractions get encoded without extra circuitry, and why shaving bits off a model's weights is never quite free.