Zero to Frontier Engineering.
A separate section, not a topic box: a fixed-order log through the transformer stack end to end, prerequisites through frontier interpretability, dense field notes instead of one post per finding.
-
Transformers, in the order they actually need to be learned
Field notes on the transformer stack top to bottom: prerequisites through frontier interpretability, with real production numbers from GPT-3, LLaMA, Mistral, PaLM, and DeepSeek.
-
Long context, fast inference, smart decoding, and life beyond attention
Field notes on the four things that actually determine whether a transformer survives contact with production: how far it can see, how fast it can serve, how cleverly it picks the next token, and whether a non-attention architecture does any of this better.
-
The LLM lifecycle: pre-training to production
Field notes on the five phases that take a pile of GPUs to a model people actually use: pre-training, post-training, adaptation, reliability/evals, and scale engineering.
-
Distributed training systems: frameworks, parallelism, sharding, and the production details that make it work
Field notes on how you train a model too large for one GPU, one node, or even one cluster, without losing correctness or efficiency.
-
One codebase, six phases, four model families: training and serving Llama, Qwen, Mistral, and multimodal models at scale
Field notes on llama-scale-infra: how one config-driven model class survives six phases of scale, and exactly where 'just change the config' stops being true.