GPUs & Hardware
What the silicon, memory, and network are actually doing.
4 logs in this box-
The roofline model: why LLM decode uses 2% of a GPU's peak, and it's not a bug
SMs, tensor cores, and HBM given their actual architectural home, the roofline equation derived to an exact ridge point, and why FlashAttention, the KV cache, and quantization are three different answers to the same underlying number.
-
How gradients cross the wire: TCP, RDMA, and why one dead link hangs 16,000 GPUs
Why standard TCP is too slow for gradient sync, how RDMA bypasses the CPU entirely, the ring AllReduce algorithm derived and shown to be bandwidth-optimal, and the real failure modes that freeze an entire cluster at once.
-
Parallel filesystems, checkpoint I/O, and the noise nobody's code caused
How striping lets hundreds of GPUs read the same dataset without one drive becoming the bottleneck, why data loading and checkpointing are opposite I/O problems, and the real incident where someone else's job quietly stole training throughput.
-
Why 'frozen' training jobs usually aren't: processes, memory, and the illusions Linux maintains
Virtual memory as an illusion the OS maintains, why fork() and CUDA don't mix, the GIL's real cost in RL training loops, and how to actually find the OOM killer's fingerprints in the kernel log.