Day 95 · The assembly line of attention

The Transformer

You will be able to
  • Draw a transformer block from memory: attention, residuals, layer norm, FFN
  • Explain why position must be injected and how positional encodings do it
  • Explain causal masking and why decoder-only models train on every position at once
  • Distinguish encoder, decoder, and decoder-only architectures and name a model for each
  • Estimate a transformer's parameter count from its config (d_model, layers, vocab)
Today's ~115 minutes
Spaced-rep warm-up: Day 94 attention cards10 min
Guided 1: Illustrated Transformer active read-along with redrawing25 min
Tech read + transformer-stack visualizer15 min
Guided 2: causal masking + shape trace + napkin math25 min
Practice: model-card estimates20 min
Project commit + quiz + flashcards20 min

Builds on: Day 94Scaled dot-product attention · Day 89Normalization & training stability · Day 85MLPs / feed-forward layers

The analogy

Yesterday you built one meeting room where every word listens to every word. A transformer is the whole office building: the same meeting-then-deskwork floor plan, stacked twelve, forty, ninety-six floors high. On each floor, tokens first hold the attention meeting (gather context from colleagues), then go to their desks for private processing — a small two-layer MLP where each token digests what it heard, alone. Meeting, deskwork, next floor.

Two pieces of office infrastructure make a skyscraper of these floors trainable. First, every meeting and every desk has a bypass corridor: a token's original vector flows AROUND each step and gets the step's output ADDED to it. If a floor has nothing useful to add, the corridor lets information pass unharmed — and, for training, it gives Day 86's blame a clean highway down ninety-six floors instead of dying in the stairwells. Second, at every doorway there's a recalibration station (layer norm) keeping each token's vector in a healthy range, floor after floor.

One oddity to fix at the entrance: the attention meeting is a round table — it has no idea who sits where. "Dog bites man" and "man bites dog" would look identical. So each token's entry badge encodes its seat number: positional information, added to the embedding before floor one.

Why this matters on the job

This architecture IS the product you will spend your career deploying: GPT, Claude, Llama, the embedding models of Day 115, the rerankers of Day 117 — all this floor plan with different configs. Reading a model card ("32 layers, d_model 4096, 32 heads") should conjure a concrete machine and an approximate parameter count in your head. Causal masking — today's subtlest idea — is why LLM training is so efficient and why generation is one-token-at-a-time, which is TTFT and tokens/sec economics when you budget latency for a customer on Day 155.

Watch it happen

The assembly line of attention — one token's trip through GPT

step 1 / 6
Token idsDay 96Embed+ positionAttentionlook aroundFFNthink alone× N layersrepeatHeadlogits
"The cat sat" → [464, 2415, 3332]

The word "sat" (as token ids) enters the line. Six stations stand between it and a prediction of the next word.

Guided practice

guided 1

Illustrated Transformer read-along, actively

25 min
  1. Open The Illustrated Transformer and read from the top through the encoder section, slowly.
  2. As you read, redraw each major figure BY HAND on paper: the QKV projections, the multi-head split, the residual+norm wrapper, the FFN. Redrawing is the active ingredient — no screenshots.
  3. When you reach the decoder section, write in the margin where the causal mask enters, and which attention layer of the decoder is cross-attention (queries from the decoder, keys/values from the encoder — Day 94's deep-dive).
  4. Close the article. Draw one complete decoder-only block from memory, labeling every arrow. Check and mark gaps in red.
  5. File the drawing — Day 98 asks you to redo it cold.
guided 2

Causal masking in NumPy + trace one token

25 min
  1. Import attention() from your attention_lab.py and add a causal=True option: build a mask where score[i][j] = -1e9 for j > i, added before softmax.
  2. Run it on Day 94's 4-token example. Verify: row 1 attends 100% to itself; row 4 is UNCHANGED from yesterday ([0.221, 0.221, 0.449, 0.109]) because token 4 could already see everyone. The weight matrix must be lower-triangular.
  3. Print the ASCII heatmap and confirm the upper triangle is blank.
  4. Now trace one token through one block on paper, shapes only, d_model = 8, seq = 4: x (4×8) → +position (4×8) → LN → attention (4×8) → +x → LN → FFN 8→32→8 → +. Write every shape at every arrow; nothing changes shape end to end — that invariant is what lets blocks stack.
  5. Parameter count for this toy block: attention 4·8² = 256, FFN 8·32·2 = 512 (ignore biases) — confirm 12·d² = 768. Then do GPT-2 small on the napkin: 12·12·768² + 50257·768 ≈ 124M.
🐍 python — editable, runs in your browser
Ctrl/⌘+Enter runs · Tab indents · numpy/pandas/sklearn auto-load on import (torch and network calls need a local run)

On your own

Model-card napkin math

20 min

Using only params ≈ 12·L·d² + vocab·d, estimate total parameters for: (a) L=24, d=1024, vocab=50k; (b) L=32, d=4096, vocab=128k; (c) your Day-97 tiny GPT: L=4, d=128, vocab=65. Then answer: (1) for model (b), what fraction of parameters is embeddings vs blocks? (2) if you double d at fixed L, what happens to block parameters? (3) which grows a model faster, doubling L or doubling d, and why?

Constraints: napkin only — no code until you have three written estimates; then verify with a 5-line script.

Hints: (a) ≈ 353M; blocks scale as d² so doubling d quadruples them while doubling L merely doubles them.

Ship before you stop

The annotated block diagram + masked attention

Commit two artifacts: (1) causal_attention.py — the masked attention with assertions that row 1 is one-hot and row 4 matches Day 94, plus the lower-triangular heatmap; (2) transformer_notes.md — your from-memory block diagram (photo or ASCII art), the per-arrow shape trace at d_model = 8, the napkin parameter math for GPT-2 small and the three practice configs, and a five-line "why residuals, why layer norm, why mask" section in your own words. These notes are the spec you will build against on Day 97 — every component you can't explain today becomes a bug you can't find tomorrow.

Rubric — check what you completed (0/6)

Common mistakes & misconceptions

  • Thinking attention does the "thinking." Attention routes information BETWEEN tokens; the FFN (two-thirds of the parameters) processes it per token. Both halves matter.
  • Forgetting position entirely — attention alone cannot distinguish "dog bites man" from "man bites dog". If a from-scratch transformer outputs order-independent nonsense, the missing position embedding is the first suspect.
  • Confusing layer norm with batch norm. LN is per-token (no batch statistics, no train/eval difference); BN's Day-89 caveats do not apply.
  • Placing the causal mask after softmax. The −∞ must enter BEFORE normalization so masked positions get exactly zero weight and rows still sum to 1.
  • Believing generation-time causality is a special mode. The mask is there during training too — that is what makes all n positions honest training examples in one pass.
  • Reading "96 layers" as 96 different designs. It is one block stamped 96 times — which is why the residual stream must keep its shape, and why your shape trace matters.
Knowledge check

Q1. Why do transformers need positional encodings?

Q2. The causal mask sets scores to −∞ for j > i BEFORE softmax so that…

Q3. Roughly how many parameters does a decoder-only model with L=12, d=768, vocab=50k have?

Go deeper — curated resources

articleJay Alammar — The Illustrated Transformer (full read-along)45 minpaperAttention Is All You Need — architecture sections (§3) with figures25 minvideo3Blue1Brown NN playlist — transformer chapters25 minreponanoGPT — read model.py top to bottom (tomorrow's blueprint)20 min
If you have a third hour
  • Learned vs sinusoidal vs rotary positionsGPT-2 learns a position table (hard context cap); the 2017 paper used fixed sinusoids; modern models (Llama-family) use RoPE, rotating Q/K by position angle. Worth a search once Day 97's build makes position embeddings concrete.
Done means
  • Block diagram reproduced from memory with gaps marked and fixed
  • Causal attention verified (one-hot row 1, unchanged row 4, lower-triangular map)
  • GPT-2 napkin math within 10% and practice configs verified by script
  • transformer_notes.md committed as the Day-97 build spec
  • Quiz ≥ 2/3
How this connects

← Back: The block wraps Day 94's attention with Day 85's MLP as the FFN; residuals are Day 86's gradient-highway lesson built into architecture; layer norm is Day 89's normalization idea, per token.

Forward →: Day 96 decides what the tokens ARE. Day 97 turns transformer_notes.md into working code, Day 100 explains how these stacks are trained at scale, Day 101 prices the n² and the context cap you met today, and Day 128's LoRA surgically edits exactly these weight matrices.

Unlocks: D96 Tokenization · D97 Tiny GPT Lab I — Build It · D98 Week 14 Checkpoint: Attention, Locked In · D100 How LLMs Are Trained