Demo Day

The presentation, delivered

1. Set up: production URL open, dashboard on a second tab, architecture diagram ready, timer visible. Audience: the best you can arrange TODAY — a friend on a call, a study group, a community demo channel, or a recording you will publish. Do not postpone for a better audience; delivered-today beats perfect-someday.
2. Deliver the eight beats. Hold the 10-minute cap — running long on your own graduation is still running long.
3. Beat 5 discipline: say the actual numbers with their intervals ("groundedness 0.93, plus or minus 0.03, on a 45-case golden set the CI re-runs on every push"). This beat is what separates you from every demo your audience has seen.
4. Beat 8 discipline: make the ask explicitly — "I'm looking for AI engineer / forward-deployed roles; the repo is at this URL."
5. Afterwards: collect one piece of feedback per audience member (what landed, what confused), and log the questions you were asked — Demo Day questions are interview questions in disguise. If recorded: publish the video link in the README today, imperfections included.

The 40-question mastery final

Closed notes. Write answers on paper, ≤ 2 sentences each. Score with the key (✓ full, half for partial), tally BY PHASE, then run the gap protocol in the practice block.

Phase 1 (D1–21): 1. A list is passed to a function that appends to it — what does the caller see afterwards, and why? (D4) 2. Name git's three areas and the command that moves work between each. (D8) 3. Why does a generator handle a 10 GB file where a list fails? (D11) 4. Structure of a good test per arrange-act-assert — and what makes a test "flaky"? (D18)

Phase 2 (D22–49): 5. Complexity of x in my_list vs x in my_set — and the classic bug this causes inside a loop. (D22) 6. What problem shape triggers "sliding window"? (D24) 7. Why does BFS (not DFS) find shortest unweighted paths? (D31) 8. Binary search "first true" — what invariant do you maintain? (D33) 9. LEFT JOIN vs INNER JOIN: which rows appear? (D37) 10. What does a B-tree index speed up, and what does it cost? (D38)

Phase 3 (D50–63): 11. What does cosine similarity measure, and why does it power embedding search? (D50) 12. What does the gradient point toward, and what does gradient descent do with it? (D53) 13. A test is 99% accurate; the disease is 1-in-10,000. Roughly what fraction of positives are real, and what is this trap called? (D59) 14. Cross-entropy in one sentence — and what does "perplexity 20" mean? (D62)

Phase 4 (D64–84): 15. Why does vectorized NumPy beat a Python loop by 100x? (D64) 16. Define data leakage and give the temporal example. (D69) 17. 99% accuracy on 1%-positive fraud data — why is this meaningless and what do you report instead? (D75) 18. How does k-fold cross-validation catch what a single split misses? (D76)

Phase 5 (D85–105): 19. What single calculus rule does backprop apply, and along what structure? (D86) 20. Why must "similar meaning → nearby vectors" hold for embeddings to be useful? (D92) 21. Walk Q/K/V through one attention step. (D94) 22. Why does BPE cause LLMs to fumble arithmetic and spelling? (D96) 23. Temperature 0 vs 1.2: what changes in the sampling distribution? (D102)

Phase 6 (D106–133): 24. Why are LLM chat APIs stateless, and what does that make YOUR job? (D106) 25. The tool-use loop: what happens between tool_use and the final answer? (D111) 26. Chunks too small vs too large — what breaks in each direction? (D114) 27. What does BM25 catch that dense retrieval misses, and how are they fused? (D116) 28. Name three agent budget/stop mechanisms and why agents need them. (D120/125) 29. Direct vs indirect prompt injection — and why is the indirect kind worse for RAG? (D132)

Phase 7 (D134–147): 30. Why do golden sets need counterfactual/negative cases, not just happy paths? (D134) 31. Three LLM-as-judge biases and one mitigation each. (D135) 32. Why must RAG evals separate retrieval from generation? (D136) 33. Two eval runs: 84% then 88% on 50 cases. What must you check before celebrating? (D139)

Phase 8 (D148–161): 34. Why does Dockerfile instruction ORDER matter for build speed? (D148) 35. What belongs between "tests pass" and "deploy" in an AI service's pipeline, and why? (D141/152) 36. TTFT vs tokens/sec — which does streaming fix, and what does the KV cache save? (D155) 37. SLO + error budget in two sentences. (D157)

Phase 9 (D162–179): 38. The Mom Test in one rule. (D163) 39. SaaS vs VPC vs air-gapped — the one-axis summary. (D170) 40. A customer reports "the AI got worse." Your first three questions, in order. (D172)

KEY (gist): 1 Caller's list mutated - one shared object, two names (aliasing).
2 Working dir -add-> staging -commit-> history. 3 Lazy: one item in memory at
a time. 4 Set up state, act once, assert result; flaky = depends on order/time/
randomness. 5 O(n) vs O(1); accidental quadratic in a loop. 6 Contiguous
subarray/substring optimizing a shrinkable constraint. 7 Explores by distance
layers, so first arrival = fewest edges. 8 Everything left of lo is false,
right of hi is true - the boundary is the answer. 9 INNER: matches only; LEFT:
all left rows, NULLs where unmatched. 10 Point lookups/range scans on indexed
columns; slower writes + storage. 11 Angle between vectors, scale-free -
"pointing the same direction in meaning-space". 12 Steepest ascent; step the
OPPOSITE way, repeat. 13 ~1% real (99/10,099); base-rate neglect. 14 Average
surprise of true labels under your predicted distribution; perplexity 20 = as
uncertain as choosing among 20 equally likely options. 15 One interpreted trip
through compiled C loops vs per-element interpreter overhead. 16 Future/target
info in training features; temporal: training on data from after the
prediction moment. 17 Predicting all-negative scores 99%; report
precision/recall/PR-AUC per the harm model. 18 Every point gets a turn as
validation; variance across folds exposes split luck. 19 Chain rule, applied
backwards over the computational graph. 20 Retrieval ranks by distance - if
geometry does not encode meaning, nearest = noise. 21 Query·Key scores ->
softmax weights -> weighted sum of Values. 22 Digits/letters merge into
opaque multi-char tokens - the model never reliably sees characters.
23 T=0 argmax determinism; T>1 flattens toward uniform - more diverse, more
error-prone. 24 Server keeps no conversation; you resend full context - so
context management is app-side engineering (D122). 25 Your code executes the
tool, returns a result message, model continues with it - possibly looping.
26 Small: fragmented, unsupported answers; large: diluted retrieval, wasted
tokens. 27 Exact identifiers, rare terms, acronyms; reciprocal rank fusion.
28 Step/token/time caps, approval gates for irreversible actions, stop
criteria - loops and drift are the default failure. 29 Direct: user attacks;
indirect: instructions hidden in retrieved DATA - RAG fetches attacker
content into the prompt automatically. 30 Systems must refuse/handle what
they can't answer; happy-path-only evals certify overconfidence.
31 Position (swap order), verbosity (pin/normalize length), self-preference
(different judge model; calibrate all vs humans). 32 Wrong-context vs
wrong-generation need different fixes; a bad answer alone doesn't tell you
which stage failed. 33 Sample size! ~50 cases -> CI ~±10 pts; difference is
within noise - run more cases or paired comparison. 34 Layer caching:
unchanged early layers reuse cache; copy code LAST so dep layers survive
edits. 35 The eval gate (golden set + thresholds): correctness of AI behavior
isn't covered by unit tests. 36 Streaming fixes perceived wait (TTFT); KV
cache avoids recomputing attention over the prefix per token. 37 SLI measured,
SLO target (e.g. 99.5% under 3 s); error budget = allowed shortfall that
gates release risk. 38 Ask about past behavior, never hypotheticals -
"would you use this?" invites lies. 39 Data control rises, your access and
observability fall, support cost multiplies. 40 Impact (what/who, quantified),
scope (what boundary), recent changes (deploys, corpus, provider, config).