Every chapter of the book, as something you can break. Poison a language model, train one from scratch, be the human who rates its answers, cheat its exams. It all runs in your browser: no AI service, no server, no tracking.Twenty experiments, each computed for real in your browser: n-gram and MLP language models, softmax temperature, BPE, gradient descent, the Chinchilla scaling law, BM25, a transformer forward pass, Bradley–Terry reward fitting, absmax quantization, PageRank and spectral clustering. Flip the lever any time for the plain-English version.
Chapter 1, and the book’s title
Poison the well
A language model doesn’t know facts. It knows what it read most often. Pick a lie, then slide copies of it into the training data. A real (tiny) language model retrains instantly and tells you what it now believes. Garbage in, gospel out.A word n-gram model with backoff, retrained from scratch on every change. The prediction is a maximum-likelihood estimate, so the decision boundary is simply where count(lie) overtakes count(truth). Neural models smooth and generalize this, but the dependence on the data distribution is the same.
Chapter 1
Turn up the temperature
Every chatbot has a hidden knob called temperature. Turn it down and the model says the most likely word every time. Turn it up and the cat ends up sitting on the tax code. Same model, same scores, one number changed.Softmax with temperature over 40 candidate tokens: pi ∝ exp(zi/T). Watch the entropy, the perplexity (effective number of choices) and the top-p nucleus move as T sweeps from near-argmax to near-uniform.
Chapter 2
The meat grinder
Before a model reads anything, your text gets chopped into chunks called tokens. Slide the merges up and watch letters fuse into syllables and then words.A real byte-pair encoder, trained in your browser on a few pages of Chapter 1. Each merge fuses the most frequent adjacent pair: .
Chapter 3
The garbage sorter
Models learn from the internet, and the internet is mostly cookie banners, spam and copies of copies. Here are 24 pages from a pretend web crawl. Turn the filters up and see how little survives.Exact dedup, near-dedup by Jaccard similarity on word 3-gram shingles (what MinHash approximates at web scale), a length floor, a heuristic junk score (caps ratio, symbol density, repetition, vowel ratio) and a boilerplate blocklist.
Chapter 4
Arithmetic with meaning
Pick an analogy and run it. The arrows are the arithmetic; the answer is whichever word sits closest to where you land.A hand-placed 2-D toy, not a trained projection. The answer is the nearest neighbour by straight-line distance, excluding the three inputs; real embeddings usually use cosine similarity.
Chapter 5
The Attention Club
Every word gets in. Not every word gets noticed. Pick a word to see who it listens to, then change the last word and watch “it” change its mind.Illustrative weights for a single head: row of . Real heads are messier and there are dozens of them per layer.
Chapter 5, in graph theory
Who listens to whom
The same attention, drawn as a social network. Every word is a dot, every arrow means “I’m paying attention to you.” Hide the quiet arrows and the sentence’s real structure shows up.Thresholded attention matrix as a weighted digraph, Aij = αij·1[αij ≥ τ]. Under a causal mask it’s a DAG. Node size is weighted in-degree; the table adds PageRank.
Chapter 6
Follow one word through the machine
Ride along with the word “the” as it goes through a complete transformer, step by step: from ticket number to vector to a guess about the next word. The machine is real. It just hasn’t been trained yet.A full pre-LN decoder forward pass computed in your browser: d = 12, two layers, one head each, GELU MLP (12→48→12), tied unembedding. Seeded random weights, so the output is noise; every shape and operation is the real one.
Chapter 6
The clock with many hands
A transformer reads every word at the same time, so how does it know which came first? Each position gets its own setting of a clock with many hands, some spinning fast and some barely moving.Sinusoidal positional encoding: PE(p, 2i) = sin(p·ωi), PE(p, 2i+1) = cos(p·ωi), ωi = 10000−2i/d. Each sin–cos pair is a rotation, so PE(p+k) is a fixed linear map of PE(p).
Chapter 7
Watch a model learn to write
This is a real neural network, starting from random noise, learning to write by reading Chapter 1 over and over. Press start. At first it babbles. A minute later it writes things that look like English and mean nothing. That’s the whole trick, just much, much bigger.A character-level MLP language model (6-character context, 10-d embeddings, 128 tanh units, ~22k parameters) trained with minibatch backprop, written from scratch in TypeScript. Switch to plain SGD or crank the learning rate to make it diverge.
Chapter 7
Roll down the hill
Training a model means one thing: roll downhill. The landscape is how wrong the model is, the ball is the model, and every step goes a little further down. Pick a starting point and a step size. Small steps crawl. Big steps leap clean over the valley. And sometimes the ball gets stuck in a dip that only looks like the bottom.Gradient descent x ← x − η∇f(x) on a non-convex surface with a global minimum, a local minimum and a ridge, with optional heavy-ball momentum. Drag the surface to rotate; click the contour map to drop the ball anywhere.
Chapter 7
The scaling-law slot machine
Why is everyone building power plants? Pick how big your model is and how much it reads. A real formula from the Chinchilla paper predicts how good it gets, and a back-of-the-envelope estimate gives you the electricity bill. Spoiler: the bill wins.The fitted Chinchilla scaling law L(N, D) = E + A/Nα + B/Dβ (Hoffmann et al., 2022) with C ≈ 6ND, the compute-optimal frontier, and published models placed on it. Cost assumes H100-class GPUs at 40% utilization.
Chapter 8
You are the reward model
Chatbots learn manners from people clicking “this answer is better.” Now you’re the people. Pick the better answer a few times, then see what kind of assistant your clicks just built.Pairwise preferences fit a linear Bradley–Terry reward model over answer traits by online logistic regression; a KL-regularized policy then chooses among candidate answers. Reward flattery and watch sycophancy appear.
Chapter 9
Win a benchmark without getting smarter
AI companies love to announce they topped a leaderboard. Here’s one way to do it: let the test questions sneak into the training data. Your model shoots up the rankings. Then give it a test it hasn’t seen.Test-set contamination, simulated with a fixed true skill p. Public score ℓ + (1 − ℓ)p climbs with the leak fraction ℓ while a held-out set stays at p.
Chapter 10
The squeeze
Big models are too big for your laptop, so people store every number with fewer digits. A little squeezing is invisible. A lot turns the model to mush. Drag the slider and watch it happen to the numbers and to a picture.Symmetric absmax quantization of 4,096 synthetic weights with outliers, per tensor or per 64-weight block, from 16 bits down to 2. Watch the outliers set the scale and crush everything else.
Chapter 11
Give it a library card
Ask a question about Chapter 1. With a library card, the system looks up the right passage first and quotes it. Without one, you get what a model does on its own: something confident and made up.Retrieval over passages of Chapter 1 with real Okapi BM25, then extractive answering from the top passage. The no-retrieval answers are written by hand to illustrate unsupported generation.
Chapter 12
The conveyor belt
A model only sees the last so-many words of a conversation. Everything older falls off the end of the belt. Shrink the window and watch the chatbot forget who you are.A sliding context window (words standing in for tokens), with the quadratic attention cost and the KV-cache memory bill that make long context expensive.
Chapter 13, in graph theory
The map of the book
Thirty ideas from the book, each pointing to the ideas it builds on. Tap “A working chatbot” to see everything it takes, or tap any other idea to trace what it needs and what needs it. The biggest circles are the ideas everything else leans on.A prerequisite DAG ranked by PageRank (the edges point from an idea to its prerequisites, so rank flows to foundations), laid out with a deterministic force simulation. Change the damping factor and watch the ranking shift.
Chapter 13, in graph theory
Six degrees of the book
Pick any two ideas in the book and see how few steps connect them. Then let the math sort all thirty ideas into groups by itself, using nothing but which ideas link to which.Breadth-first shortest paths on the idea graph, and spectral clustering: the first k eigenvectors of the normalized Laplacian L = I − D−½AD−½, row-normalized, then k-means. Eigenvalues and λ₂ shown.
Bonus, from the series “The Chip That Says No”
The chip that says no
An AI agent is allowed to pay one invoice. Try to make it do more. A software guard that reads the request can be sweet-talked. A chip that only compares numbers can’t.A software guardrail versus a hardware comparator at the point of action: the request must match a fresh, signed ticket bound to that exact operation, with an unused nonce, within budget, under the current policy epoch, and policy changes need k-of-n human signatures.