Week 15 Checkpoint: Phase 5 Assessment
- Recall the core mechanisms of Phase 5 from memory: attention, transformers, tokenization, training, decoding
- Score a cumulative self-assessment and map every miss to a revisit day
- Write a one-page "how an LLM works" explainer a non-technical stakeholder can follow
- Consolidate the week's artifacts into a coherent Phase 5 portfolio folder
| Spaced-rep: full week-15 deck drill | 15 min |
| Guided: closed-book recall drills + diff | 30 min |
| Guided: 25-question phase assessment + scoring | 25 min |
| Practice: re-derive the generation loop | 15 min |
| Project: stakeholder explainer + Q&A test | 30 min |
| Consolidate artifacts, log scores, plan repairs | 10 min |
Builds on: Day 99 — Tiny GPT training · Day 100 — Training pipeline · Day 102 — Decoding & sampling · Day 104 — Context & hallucination
Exam day — but the exam you want to take. For three weeks you have gone from a single artificial neuron to training your own GPT and cataloging the failure modes of trillion-token models. Today does two things that cement it. First, retrieval practice: closing the notes and pulling explanations out of your own head is what converts three weeks of exposure into durable knowledge — testing is not measurement of learning, it IS learning, which is why every seventh day of this program works this way.
Second, translation practice. The Feynman test says you understand something when you can explain it simply without lying. Your deliverable is a one-page explainer of how an LLM works, written for a smart executive who has never seen a matrix. This is not a soft skill bolted onto the technical work — for a forward-deployed engineer it IS the work. The person who can hold the internals in their head AND make a VP genuinely understand "it predicts the next word, and here is why that is both amazing and risky" is the person who gets trusted with the project.
Phase 5 is the substrate for everything after it: Phase 6 calls these models, Phase 7 evaluates them, Phase 8 serves them. Gaps here compound — a fuzzy grasp of context windows becomes a bad RAG design in two weeks. The stakeholder explainer also becomes a real professional asset: FDE interviews almost universally include "explain LLMs to a non-technical audience," and the written version you produce today is a portfolio piece you will reuse in the Day-168 simulation and actual customer decks.
Guided practice
Closed-book recall drills
30 minClose every note and editor. On paper or a blank file:
- Write the scaled dot-product attention formula and label every symbol and shape (5 min). Then explain in two sentences what Q, K, V mean as a soft lookup.
- Draw the three-stage training pipeline with data types and rough scales at each stage (5 min).
- List the decoding strategies with their mechanics: greedy, temperature, top-k, top-p, repetition penalty (5 min).
- Write the five hallucination types with one trigger each (5 min).
- State: the Chinchilla rule, why attention is O(n²), the expected initial loss for vocab size V, and the lost-in-the-middle finding (5 min).
- Now open your notes and diff. Mark each item green (correct), yellow (partial), red (missed). Reds go into tomorrow's flashcard deck with the revisit day noted (5 min).
Cumulative self-assessment (25 questions)
25 min- Take the Phase 5 assessment from the app (25 questions spanning D85-D104), closed book, timed at 20 minutes.
- Score it. For each miss, write the day to revisit — the question metadata carries it.
- For your two weakest clusters, do a 5-minute targeted re-read of that day's tech section NOW, while the miss is fresh.
- Log your score in your progress tracker next to the Day-91 and Day-98 checkpoint scores — the trend matters more than any single number.
On your own
Re-derive from an empty file
15 minFrom memory, in a fresh Python file: write the skeleton of the generation loop — crop context to block_size, forward pass, take last-position logits, apply temperature, softmax, sample, append. Pseudocode-level fidelity is fine; then diff against your actual Day-102 decoding_lab.py and note anything you forgot (the cropping and the renormalization are the classic omissions).
Hint: if you can write this loop cold, you can whiteboard "how does an LLM generate text" in any interview.
The stakeholder explainer
Write how-an-llm-works.md: one page, for a non-technical executive. Requirements: one analogy sustained throughout; covers prediction objective, training data and cutoff, why capability emerges, why hallucination is structural, and two concrete business implications (ground it with retrieval; evaluate before trusting). Read it aloud once — sentences you stumble over get rewritten. Then the FDE test: have someone non-technical (or an LLM role-playing a skeptical CFO) read it and ask three questions; append a Q&A section with your answers. Commit to the portfolio repo — this document goes to real customers eventually.
Common mistakes & misconceptions
- Reviewing by re-reading instead of recalling. Recognition feels like knowledge; only closed-book retrieval reveals and repairs the gaps.
- Writing the explainer for other engineers. If your test reader asks "what is a parameter?", you have not met the constraint — translate or cut.
- Overclaiming in the explainer ("it understands", "it reasons like a human") or underclaiming ("it is just autocomplete, it knows nothing"). Both destroy credibility; the honest middle is the skill.
- Skipping the miss-to-revisit mapping. An unscored assessment is a quiz; a scored one with targeted repairs is spaced repetition working as designed.
- Treating the checkpoint as pass/fail. It is a diagnostic — a 17/25 with a repair plan beats an unexamined 21/25.
- Letting the week's artifacts scatter. Ten minutes of consolidating repos and notes now saves an hour when Day 119's capstone kickoff asks you to draw on all of it.
Q1. In scaled dot-product attention, why divide the scores by sqrt(d_k)?
Q2. Which stage of the pipeline is responsible for the overwhelming majority of a model's knowledge?
Q3. Your explainer must state why LLMs sometimes invent facts. The honest one-sentence version is…
Go deeper — curated resources
- Teach it to cement it — Record a 5-minute screen video walking through your tiny GPT repo as if onboarding a new teammate. Explaining code you wrote is the strongest retrieval practice available.
- Recall drills done with green/yellow/red diff recorded
- Assessment scored with every miss mapped to a revisit day
- Explainer committed and tested on a non-technical reader with Q&A appended
- Phase 5 artifacts (tiny GPT repo, matrices, guides) consolidated in one place
- Weak-cluster repairs scheduled into next week's warm-ups
← Back: This checkpoint closes the arc from Day 85's single neuron through Day 104's failure taxonomy — three weeks from "what is a weight" to "I trained one and know its limits."
Forward →: Phase 6 starts tomorrow: Day 106 makes your first structured API calls to a frontier model, and everything you consolidated today becomes the mental model behind every request. The explainer resurfaces in the Day-168 FDE simulation and your interview prep on Day 179.
Unlocks: D162 The FDE Role · D163 Customer Discovery & the Mom Test · D167 Demo Craft