Testing II & Debugging
- Use fixtures to share setup, and tmp_path to test file-writing code safely
- Replace an environment dependency in a test with monkeypatch, and say when mocking is a smell
- Read a traceback bottom-up and locate the true failing frame
- Drive pdb via breakpoint(): inspect variables, step, and continue
- Convert any fixed bug into a permanent regression test
| Spaced-rep warm-up: due flashcards | 10 min |
| ELI5 + tech read: fixtures, tracebacks, pdb, regression tests | 20 min |
| Guided: tmp_path/monkeypatch tests + the pdb tour | 35 min |
| Practice: bisect the mystery | 20 min |
| Project: harden the analyzer suite to 15+ tests | 25 min |
| Quiz + flashcards | 10 min |
Builds on: Day 18 — pytest fundamentals · Day 5 — Files & exceptions
When a detective arrives at a crime scene, the amateurs are already trampling it — moving furniture, guessing suspects, changing things at random. The professional does something slower and far faster: cordon off the scene, photograph everything exactly as found, dust for prints, and only THEN form a theory. Debugging is the same discipline. The traceback is the photograph — a complete record of where the program died and every call that led there; you read it before touching anything. The debugger (pdb) is the fingerprint kit — it freezes the program mid-crime so you can inspect every variable as it actually was, not as you assume it was. Bisection is narrowing the suspect list: cut the possibilities in half, test, repeat.
And today's testing tools are the forensics lab itself: fixtures are prepared, sterile workbenches every analysis starts from — same setup, every time, torn down after. tmp_path hands each test a disposable room so file experiments can't contaminate the real house. When the case closes, one rule separates professionals from amateurs: every solved crime becomes a regression test — an alarm rigged so this exact crime can never be committed silently again.
Interview reality: you will spend far more professional hours debugging than writing fresh code, and LLM systems make it harder — failures are often buried in glue code, not the model. The discipline transfers directly: reading a traceback calmly is the same skill as reading a failed trace on Day 142; bisection is how you find which prompt change broke an eval on Day 141; "reproduce, then fix, then rig the alarm" is exactly the log-to-eval-case flywheel of Day 143. FDEs debug in customer environments with limited access (Day 172) — method, not luck, is all you get to bring.
Guided practice
Fixtures + tmp_path: testing the I/O shell
20 min- In
tests/test_io.py, write thesample_logfixture from the starter code, then tests for your analyzer's file-facing functions: reading a log file, and the missing-file behavior. - Terminal: run
pytest tests/test_io.py -q— note each test got its own fresh tmp_path; print it withp = tmp_path; print(p)once to see where pytest puts it. - Add the monkeypatch test from the starter: it sets the
ANALYZER_LOG_LEVELenv var and asserts your Day 16get_log_levelprecedence function honors it — without polluting your real shell. - Add a teardown demo: a fixture with
yieldthat prints "setup" before and "teardown" after; run withpytest -sonce to watch the order. Delete the prints after.
The pdb tour of a real crash
15 min- Create
crime_scene.pyfrom the starter — it crashes. Terminal: runpython crime_scene.pyand read the traceback BOTTOM-UP: name the exception, the raising line, and the last frame in "your" code before looking further. - Add
breakpoint()on the line above the crash site and rerun. You are now in pdb at the frozen scene. - Interrogate:
p record,p record.get("status"),lto see surrounding code,nto step one line. Find the flawed assumption (some records have no status key). - Quit (
q), fix the function to handle the missing key, remove breakpoint(), and rerun clean. - Now the professional finish: write the regression test
test_summary_handles_record_without_statusin your suite, watch it pass, and note it would have FAILED before your fix (check by temporarily reverting).
On your own
Bisect the mystery
20 minDownload nothing — write mystery.py: a pipeline of six small functions chained together (normalize -> split -> filter -> parse -> aggregate -> format), where you deliberately plant one subtle bug in a function chosen by a coin flip (e.g. an off-by-one slice, a swapped comparison). Wait one hour if you can, or better: have the AI tutor plant the bug for you in a version you don't read.
Goal: find the buggy stage using bisection ONLY — check the intermediate value at the pipeline's midpoint, decide which half is guilty, and repeat. You get a maximum of 3 inspection points before naming the stage. Then confirm with pdb, fix, and write the regression test.
Hints: printing (or asserting on) the midpoint value tells you if corruption happened before or after it — that single bit halves the search space, exactly like Day 22's binary search will.
Harden the analyzer suite
Extend your analyzer's test suite to cover the I/O shell: file reading via tmp_path fixtures (valid file, file with malformed lines, empty file, missing file), env-var config via monkeypatch (both precedence directions), and at least one regression test reproducing a real bug you have personally hit since Day 14 — named test_regression_<description>, with a one-line comment citing the original symptom. Refactor anything that resists testing (a function that opens files AND computes — split it). Target: 15+ total tests, all green, no test touching the real filesystem outside tmp_path. Commit. This suite is a Day 21 shipping requirement, and its green run becomes a CI gate on Day 152.
Common mistakes & misconceptions
- Reading a traceback top-down and fixating on the first frame (often framework code). Read bottom-up: exception line first, then the deepest frame in YOUR code.
- Guess-editing before reproducing. If you cannot trigger the bug on demand with a minimal input, you cannot know your fix fixed it.
- Testing file code against real project files. They change, tests break mysteriously, and a buggy test can overwrite real data — tmp_path exists precisely for this.
- Mocking your own internals to force a test to pass. Boundary mocks (env, network, clock) are healthy; internal mocks mean the design needs splitting, not the test more magic.
- Deleting the breakpoint()-driven insight without capturing it. The bug you just fixed is the cheapest test case you will ever get — rig the alarm while the scene is fresh.
- Leaving breakpoint() calls in committed code. They freeze production processes; grep for breakpoint before every commit (a pre-commit hook can do this for you).
Q1. A test needs a log file on disk. The right approach is…
Q2. Where do you look FIRST in a long traceback?
Q3. You fixed a bug found in production. What turns that incident into permanent protection?
Go deeper — curated resources
- git bisect — When a bug appeared "sometime in the last 40 commits," git bisect binary-searches history: mark one good and one bad commit and it checks out midpoints until the guilty commit is found. Try it on your repo after Day 20 gives you more history.
- I/O shell tested end to end via tmp_path; env precedence via monkeypatch
- crime_scene.py debugged with pdb and its bug preserved as a passing regression test
- Mystery pipeline bug found within 3 bisection probes
- Suite at 15+ green tests, committed; quiz ≥ 2/3
← Back: Day 18 taught alarms for pure logic; today the I/O shell from Day 5 and the config code from Day 16 got their alarms too. The bisection method is Day 22's O(log n) insight applied to detective work.
Forward →: Day 21 ships only when this suite is green. git bisect (same idea across commits) appears when history gets long; on Day 141 a failing eval is bisected across prompt changes, and Day 172 runs this whole method inside a customer's locked-down environment.