This blog serves as a collection of my personal study notes, toy experiments, and the 'aha!' moments I encounter along the way. Please reach out if you have questions, feedback, or just want to connect.
Hi, I'm Doug 👋🏻
- Why Tokens Are EnoughModern language models don't train on text — a tokenizer chops raw text into chunks, and the model only ever sees those chunks. This indirection raises two natural questions. First: what does...tokenization information-theoryMar 16, 2026 · 9 min read
- The Hidden Variance Reduction in Diffusion LossThe variational perspective formulates diffusion models as latent variable models (LVMs) trained by maximizing the evidence lower bound (ELBO). However, standard derivations of the diffusion ELBO...diffusion variance-reductionDec 06, 2025 · 10 min read