Week 13 Checkpoint — The First Neural Check
- Reproduce the week's core mechanics from memory: forward pass, backprop rules, the five-beat loop
- Write a correct "how backpropagation works" explanation unaided, then diff it against your Day-86 notes
- Refactor the MNIST lab into the Day-82 project template with configs and a clean module layout
- Clear the week's flashcard deck and re-quiz the topics you missed
| Flashcard deck: all due cards from Days 85–90 | 15 min |
| Drill 1: backprop + loop from a blank page, then diff | 25 min |
| Drill 2: rapid-fire recall set, scored | 15 min |
| Practice: 20-minute cold-start rebuild | 20 min |
| Project: refactor MNIST lab into the template | 35 min |
| Cumulative quiz + schedule revisits for misses | 10 min |
Builds on: Day 85 — Neurons & forward pass · Day 86 — Backprop & micrograd · Day 88 — Training loops · Day 90 — MNIST lab
A week ago a "neural network" was a diagram in other people's blog posts. Seven days later you have hand-computed a forward pass, written an autograd engine, and trained a convolutional network to read handwriting at 98.5% accuracy. That is an enormous amount of new circuitry in your head — and untended, half of it will be gone by Friday. Today is maintenance day, and maintenance means retrieval, not rereading.
The forgetting curve is steep but honest: every time you successfully drag a memory out with effort — closed book, blank page — the curve flattens. Rereading feels productive and does almost nothing; recalling feels uncomfortable and does almost everything. So today you write backprop from memory and only THEN open Day 86 to diff. You recite the five-beat loop with your editor closed. You re-derive parameter counts on paper. The second half is a different kind of consolidation: refactoring the MNIST lab into your standard project template — because restructuring code you wrote two days ago forces you to re-understand every line, and it leaves you a clean scaffold the GPT labs will reuse.
Week 14 (attention, transformers, GPT) assumes this week is load-bearing: attention is dot products flowing through the same autograd, and Day 97's model trains with the same five beats. Any gap now compounds. There is also a career muscle here — "explain backprop" is a top-five interview question for AI roles, and the difference between having read it and having WRITTEN it from memory is exactly what an interviewer hears. The refactor drill trains the notebook-to-module skill that separates lab work from shippable code.
Guided practice
Drill 1 — backprop from a blank page
25 min- Close every editor tab and note. On paper or a blank file, write "How backpropagation works" in two registers: five sentences for a smart non-engineer, then a technical version with the local rules for +, ×, and tanh, why gradients accumulate with +=, and why order must be reverse-topological.
- Still from memory: write the Value.__mul__ method including its _backward.
- NOW open Day 86 and your micrograd_mine.py. Diff honestly. Mark every omission in red — each red mark becomes a flashcard.
- Grade yourself: 0–2 misses = solid; 3–5 = revisit Day 86's tech section tonight; 6+ = redo guided 1 of Day 86 tomorrow morning.
- Repeat the exercise for the five-beat loop: write it from memory with the two mode toggles, then diff against trainer.py.
Drill 2 — rapid-fire recall set
15 minAnswer on paper, then check (answers in your Week-13 notes):
- Parameters in a 784-256-10 MLP? In a 3×3 conv layer, 1→16 channels, with bias?
- z = [-3, 0.5, 2] → ReLU → ? → and the local gradient at each entry?
- dL/da for c = a·b, a = 4, b = -0.5, dL/dc = 2?
- Initial cross-entropy loss for a 26-class problem, and why?
- Which two layers behave differently under model.eval(), and how?
- Your model's val loss bottomed at epoch 12 of 50 — which checkpoint ships, and what two treatments do you try next?
Score /6 in your error log; anything missed maps to a revisit day (85, 86, 88, 89).
On your own
The cold-start rebuild
20 minIn a fresh directory with only PyTorch installed, rebuild the two-arcs classifier (Day 88) COMPLETELY from memory in 20 minutes: data synth, model, five-beat loop, val accuracy print. No peeking at trainer.py.
Constraints: timer visible; whatever is unfinished at 20 minutes gets finished with the reference open, and every line you needed to look up gets written on a flashcard.
Hint: this is the exact fluency bar interviews and incident channels demand — the loop should flow out of your fingers by now.
MNIST lab, production-shaped
Refactor mnist-lab/ into the Day-82 template: src/data.py (transforms + loaders, defined once), src/models.py (MLP, SmallCNN), src/train.py (config-driven entry point), src/predict.py (imports the SAME transform from data.py), config.yaml (or a dataclass) for all hyperparameters, and tests/ asserting the two sanity rituals plus a predict round-trip on a bundled sample image. Update the README with the week's headline numbers. Finish by re-running one ablation purely as a config change — zero code edits — to prove the structure earns its keep.
Common mistakes & misconceptions
- Rereading notes instead of retrieving. Recognition feels like knowledge; only effortful recall builds the durable trace — blank page first, notes second.
- Skipping the diff step after writing from memory. The diff IS the lesson: unmarked gaps stay gaps.
- Refactoring by copy-paste without re-running the tests and targets. A refactor that was never re-validated is a rewrite with extra confidence.
- Leaving two copies of the normalize transform alive after the refactor. The whole point is one source of truth for the train/inference contract.
- Treating a bad rapid-fire score as failure. It is the system working — each miss is now scheduled for repair instead of silently compounding into Week 14.
- Grinding new material today "to stay ahead". Consolidation day consolidates; attention arrives tomorrow either way.
Q1. In your micrograd engine, why must _backward functions run in reverse topological order?
Q2. Which sequence correctly validates during training?
Q3. Your CNN's val accuracy dropped 1.5 points after the refactor, with identical config and seed. The first suspect is…
Go deeper — curated resources
- Karpathy — Becoming a Backprop Ninja (makemore part 4) — If drills felt easy: this Zero-to-Hero installment backprops through a full MLP by hand at the tensor level, no autograd. The graduate version of Day 86.
- Backprop writeup graded with ≤ 5 red marks (or revisit scheduled)
- Rapid-fire ≥ 5/6 (misses added to the error log with revisit days)
- Cold-start rebuild reached a training loop within 20 minutes
- Refactored lab passes tests; ablation-by-config demonstrated
- Quiz ≥ 2/3; deck cleared
← Back: Everything drilled today was built on Days 85–90; the refactor target is Day 82's template; the error-log ritual continues the Day-28 interview log.
Forward →: Tomorrow embeddings open Week 14, and every day of it leans on this week: attention (Day 94) is dot products + softmax through autograd, the GPT (Day 97) is nn.Modules in your refactored template shape, and its training (Day 99) is trainer.py verbatim.