Week 2 Checkpoint: Log Analyzer
- Recall Week 2 material from memory: git, classes, dataclasses, generators, decorators, regex
- Rebuild the week's core patterns on a blank page and diff against reality
- Build an object-oriented, streaming log analyzer from a spec, reusing Day 13's parser
- Ship it with a clean, story-telling git history
| Deck drill: Weeks 1–2 cards, misses re-drilled, count recorded | 15 min |
| Blank-page protocol: five prompts + diff + FORGOT harvest | 10 min |
| Warm-up: regenerate log, smoke-test parser, sketch components | 10 min |
| Git story check: draft the commit plan | 10 min |
| Project: build the analyzer to spec | 55 min |
| Cumulative quiz + analyst's summary + revisit scheduling | 15 min |
Builds on: Day 8 — Git (clean history is graded) · Day 10 — Dataclasses and composition · Day 11 — Generator pipelines · Day 13 — logparse.py (today's engine)
Second checkpoint on the trail — and this one has a twist. At the first checkpoint you repacked the bag you'd been carrying for a week. This week you didn't just collect more gear; you built *machines*: a time machine (git), blueprints (classes), conveyor belts (generators), gift-wrap (decorators), and a sketch-artist's eye (regex). Machines rust differently than facts. You don't check a machine by reciting its manual — you check it by *running it on a real job* and seeing what grinds.
So today's checkpoint is a real job: a server has been writing a log for weeks — thousands of lines, most routine, some alarming, some junk. Your task is the analyst's: stream through it without loading it whole (belts), parse each line into structured objects (blueprints + the sketch artist), and produce the report a human actually wants — which errors dominate? which IPs are hammering us? — with a commit history that tells the story of how you built it (time machine). Every machine from the week, one assembly. First, though: flashcards and a blank page. Recall before reference — the checkpoint rule, forever.
This project is a rite of passage because it's REAL — "analyze this log and tell me what's wrong" is a task you will do professionally dozens of times, including under incident pressure with a customer watching (Day 172). It's also the week's material proving it composes: classes holding parsed data, generators keeping memory flat, regex doing extraction, git making the work reviewable. That composition is what interviews probe with take-home projects, and what Day 21 will formalize into a shipped, tested package. The recall drills matter just as much: Week 2's abstractions (closures, protocols) decay fastest without retrieval.
Guided practice
Deck drill + blank-page protocol
25 min- Flashcards first: every due card from Weeks 1–2, answered aloud before flipping. Two piles, honest hesitation counts as a miss. Re-drill the miss pile twice. Record the count next to Day 7's in journal.md — the trend is the point.
- Blank page: close everything. In recall2.py write from memory the five prompts from the tech section. Run what's runnable; let errors teach.
- Diff against your real files from the week. Every discrepancy becomes a # FORGOT line; copy them to the journal. Common finds: forgotten functools.wraps, missing default_factory, the r-prefix on patterns, diff vs diff --staged.
- For your two worst FORGOT items, write the pattern out correctly one extra time, by hand.
- Time-box the whole drill to 25 minutes — imperfect recall harvested beats perfect notes reread.
Warm up the machines — parser smoke test
10 min- Regenerate the big log: bump make_log.py to 200_000 lines and run it. Append 3 junk lines by hand (echo "not a log line" >> app.log — Day 6 reps).
- Smoke-test yesterday's engine before building on it: python logparse.py should run its demo — first five parsed dicts plus the junk count. If anything fails, fix logparse.py FIRST and commit the fix separately ("Fix logparse junk handling for empty lines" or similar).
- In the REPL, time one full streaming pass: import your parser, run sum(1 for _ in parse_lines(open("app.log", encoding="utf-8"))) — note the seconds and the fact that memory stayed flat.
- Sketch the analyzer's three components on paper with arrows: path -> LogReader.entries() -> Report.consume() -> render(). Thirty seconds of diagram saves thirty minutes of tangle — a habit that scales all the way to Day 165's architecture docs.
On your own
Git story check
10 minBefore starting the project, rehearse the history you intend to write. In your journal, draft the 4–6 commit messages you EXPECT to make for the analyzer, in order, each finishing "if applied, this commit will…". Then, as you build, hold yourself to committing at those seams (adjusting is fine; the plan is the practice).
After the build: run git log --oneline and compare against the plan. Score yourself: did any commit mix two stories? Would a stranger follow the sequence? Fix nothing retroactively today — just observe and note one improvement for Day 21's shipped project.
Hints: the natural seams are the component boundaries — that's not a coincidence; commits and components both follow "one responsibility".
The log analyzer
Build the analyzer exactly to the tech-section spec: LogEntry dataclass, streaming LogReader composed around Day 13's parse_lines, an accumulate-once Report with counts by level, top-5 error messages, top-5 IPs, junk count, and an aligned-text render; CLI entry taking the log path, run wrapped in @timed; verified on a 200k-line log with hand-planted junk. Then the analyst's minute: read your own report and write three sentences in journal.md answering "what would I tell the team?" (which error dominates, which IP looks suspicious, what you'd investigate next). The tool is the deliverable; the reading of it is the job. Commit history per the plan — it is graded as part of the project.
Common mistakes & misconceptions
- Building first, reviewing later ("the project is the review"). The recall drills are the review; the project is the integration test. Skip the drills and Week 2's abstractions quietly rot until Day 21 exposes it.
- Materializing the stream for convenience (entries = list(reader.entries()) "just to check the length"). One reflex list() and constant memory is gone. Count as you consume.
- Consuming the generator twice — once for levels, once for IPs — and getting empty second results (Day 11's exhaustion rule). One pass, all accumulators updated together.
- Re-compiling the regex per line, or worse, re-writing the pattern inline instead of importing logparse.py. You built the engine yesterday precisely so today composes it.
- One giant analyze() function instead of Reader/Report components. It works — and it's unreviewable, untestable (Day 18 will want components), and un-commit-able at clean seams.
- Commit messages written after the fact as "add analyzer stuff". You drafted the story in advance; the history should read like it.
Q1. Report needs level counts AND top IPs from reader.entries(). Correct structure?
Q2. LogEntry(**d) where d = {"date": ..., "time": ..., "level": ..., "ip": ..., "message": ...} does what?
Q3. Your word-boundary pattern works in the REPL but never matches when moved into the script as "\berror\b" (no r prefix). Why?
Go deeper — curated resources
- collections.Counter — the accumulator you hand-rolled — Counter(seq).most_common(5) replaces your counts-dict-plus-sorted dance. You built it by hand first on purpose; from Day 15 onward, use the stdlib version and enjoy knowing what's inside.
- Flashcard miss count recorded and compared to Day 7
- recall2.py written blind; FORGOT lines harvested to journal
- Analyzer passes all six rubric checks on the 200k-line log
- History matches the drafted commit plan (or the deviation is explained in journal)
- Quiz ≥ 2/3 with revisit days scheduled for any miss
← Back: One build, five machines: Day 8's git telling the story, Day 10's dataclass-and-composition structure, Day 11's single-pass streaming, Day 12's @timed on the run, Day 13's parser imported whole. Even the log came from Day 6's workshop.
Forward →: Week 3 turns this artifact professional: Day 15 refactors it with types and clean-code rules, Day 16 gives it real logging and argparse, Day 18–19 test it, and Day 21 ships it as an installable package on GitHub — the analyzer is your portfolio seed. The streaming-parse-report shape returns at scale in Day 143's log mining.
Unlocks: D15 Clean Code & Type Hints · D16 Logging, Config & CLI Ergonomics