A guided walk · free

Worlds.

One idea per world, laid out as a place you walk through by scrolling. I'll guide you. Everything you see was measured from something we actually ran.

Amit, your guide

Hi, I'm Amit. Each world here is one idea you walk through by scrolling: the camera moves, the drawing changes, and I explain as we go. Let's start with season one. We'll follow a single sentence through a chatbot.

Start at stop 1: How a Sentence Gets Chopped →

Free · no account · works on a phone, best on a laptop

Season one · 6 stops

Inside the chatbot

Follow one sentence through a chatbot, from the moment it's chopped into pieces to what changes as models grow.

  1. Stop 1 · Tokenization · A print shopHow a Sentence Gets Chopped“Before a model reads a single word, your sentence gets cut into pieces. We'll watch it happen, and you'll see why counting the r's in strawberry is harder than it looks.”What's real: A byte-pair press we trained on four novels, plus the real tokenizers behind GPT-2 and GPT-4o.
  2. Stop 2 · Next-token prediction · A hillsideLearning to Write“The whole trick behind a chatbot is guessing the next piece, over and over. We trained a tiny model from scratch, so you can walk down its real learning curve and read what it wrote at every stage.”What's real: A 1.8 million parameter model we trained on four public-domain books, sampled at every checkpoint.
  3. Stop 3 · Attention · Two river crossingsLetting the Translator Look Back“Here's the idea that made modern models possible. One translator has to squeeze a whole sentence into a single note. The other is allowed to look back. Watch what happens as sentences get longer.”What's real: Two translators we trained ourselves, with and without attention, tested on sentences up to 41 words.
  4. Stop 4 · Temperature and sampling · A river deltaHow It Picks the Next Word“A model doesn't know what comes next. It has odds for every possibility. You'll turn the temperature dial yourself and see the odds reshape, from safe and repetitive to wild.”What's real: Real probabilities from GPT-2, and 30 prompts written five different ways.
  5. Stop 5 · The KV cache · A reading roomWhy It Doesn't Reread Your Chat“Every new word looks back at everything before it. So why doesn't a long chat grind to a halt? Because the model keeps notes. We timed it with the notes and without them.”What's real: GPT-2 writing the same 1,000-piece reply both ways on a laptop, plus the published config of a current open model.
  6. Stop 6 · Emergent abilities · A ridgeEmergence, or a Change of Ruler“Bigger models seem to pick up skills out of nowhere. We trained 40 small models on addition and scored the same answers two ways. Some of the jump is the ruler. Some of it is real.”What's real: A family of eight model sizes, five training runs each, scored on 2,000 sums they never saw.
Prequel · before stop 3 · A track through a sentenceHow an LSTM Remembers“Before attention, models carried memory along a belt, word by word. This is the one to walk before stop 3 if you want the full story.”Bonus world · agents · A deskThe Run Sheet“Not about how models work inside, but about what you build around them. We follow one real agent run around a desk, from the failing test that starts it to the check that catches a bad fix.”

How to walk a world: just scroll. Every position is a frame, so you can stop anywhere and look closely. The small map in the corner shows where you are, and the sources are listed at the bottom of every world.