Capstone Hardening
- Close the D176-assigned punch-list items: auth, rate limiting, and input validation at every entry point
- Replace every leaky or vague error path with honest, actionable failure messages
- Complete structured logging coverage and clean the configuration surface
- Leave the repo in a state where a stranger could run it from the README alone
| Punch-list review + spaced-rep warm-up | 10 min |
| Read: three-pass plan for the day | 10 min |
| Pass 1: perimeter (auth, limits, validation) | 25 min |
| Pass 2: honest failure modes | 25 min |
| Pass 3: config, logging, README truth | 25 min |
| Re-run golden set + happy path; commit; quiz | 20 min |
Builds on: Day 161 — Production cutover · Day 45 — Web service architecture & hardening · Day 175 — The frozen punch list
When a building is "finished," the builder and the owner walk every room with a punch list: the door that sticks, the outlet with no cover, the window that whistles in wind. None of these stop the building from standing — and every one of them is what the owner will actually touch every day. Punch-list week is where a structure becomes a building people trust.
Your capstone stands: it answered questions in production back on Day 161. This week is the walk-through with the owner's eyes. Today is the doors-and-locks day: can anyone walk in without a badge (auth)? Can one visitor hog every elevator (rate limits)? If someone shoves garbage through the mail slot, does it jam the building or bounce politely (validation)? And when something does break, does the building say "ERROR 500" — or does it say which door stuck, what you can do now, and who has been notified? The difference between a demo and a product is almost entirely in these unglamorous rooms — which is exactly why Day 177's inspector and Day 180's audience will look there first.
Hiring managers who open candidate portfolios see a hundred demos that collapse at the first malformed request; a repo whose error paths, auth, and logs are visibly production-grade reads as "this person has operated software," which is the actual hiring signal. And the skills rehearsed today — hardening an AI service's entry points, writing honest failure messages for LLM-specific errors — are precisely week one of any FDE deployment, where the customer's pen-tester, not a reviewer, does the walking.
Guided practice
Pass 1 — perimeter: auth, rate limits, validation
25 min- Write the perimeter test FIRST: for every route, assert 401 with no/expired/wrong-audience tokens (parametrize over your route table — Day 18's pytest muscle). Run it; fix every route that fails open.
- Add per-user rate limiting to the
/askand/ingestpaths from the starter pattern. Verify: burst 20 requests as one user → 429 with Retry-After; a second user is unaffected. - Validation sweep: query length cap (chars AND a token estimate), ingest content-type allowlist, metadata filter keys allowlisted against a schema. Each rejection returns 422 with the field named.
- Commit per sub-item with punch-list references in the messages.
Pass 2 — honest failure messages
25 min- Build the failure-mode table in
docs/failure-modes.md: five rows minimum (provider timeout, provider rate-limit, retrieval empty, low-groundedness answer, vector DB unreachable). Columns: stable code · user sees · system does (fallback? retry? refuse?) · log fields. - Implement each: replace blanket except-blocks with typed handlers that emit the table's exact user message and log line. The refusal path matters most: when groundedness fails, the honest "I can't answer that reliably from the current docs — here's what I searched" beats a confident guess. Wire it to return the searched-sources list.
- Break things to verify: point the vector DB at a dead port, set the provider key invalid, ask an unanswerable question. Confirm each produces its table row exactly — message, status, log line with trace id.
- Commit with the table; Day 177 re-runs these three breaks as gates.
On your own
Pass 3 — config, logging, README truth
25 minNo steps provided — you know this codebase. Goals: (1) one config module, .env.example regenerated and complete, zero secrets in defaults or git history (check!); (2) structured log events on every request path with the consistent field set your dashboard expects — grep your logs after a manual session to prove no path is silent; (3) README executed literally top-to-bottom in a fresh clone + fresh containers; fix every drifted command.
Constraints: frozen list only — new feature ideas go to docs/backlog.md; timebox hard at 25 minutes, unfinished items return to the punch list explicitly rather than silently.
Hints: git log --diff-filter=A -- .env answers "was a real .env ever committed"; the README lie is usually in the FIRST command (a rename you forgot).
Punch-list execution: hardening day
Execute every D176-assigned punch-list item through the three passes: perimeter (auth on all routes verified by tests, rate limiting live, validation strict), honest failure (five-row failure table implemented and break-tested), and surface (config clean, logging complete, README true in a fresh clone). Update docs/finale-punch-list.md marking each item DONE with its commit hash, or explicitly bounced to D177/D178 with a reason. The repo should end today runnable by a stranger and resistant to a hostile one — tomorrow's inspector assumes both.
Common mistakes & misconceptions
- Assuming auth covers every route because it covers most. The forgotten /metrics, /docs, or debug endpoint is the classic finding; the parametrized perimeter test exists because assumption is not verification.
- Rate limiting by IP instead of user. Enterprise users share egress IPs (Day 170) — one office would rate-limit itself into an outage; key the bucket on authenticated user id.
- Failure messages that leak internals. Stack traces, provider names, and infrastructure hostnames in user-facing errors are reconnaissance gifts (Day 159); the stable code + honest message + rich LOG line separates audiences correctly.
- Making the refusal path so unpleasant users prefer the hallucination. "I can't answer reliably" must come with what WAS searched and what to try — a dead-end refusal trains users to distrust the product's honesty.
- Polishing new features during punch-list week. Every mid-flight idea that jumps the frozen list steals verification time from tomorrow's gates; backlog it — the discipline is the deliverable.
- Testing the happy path after hardening the sad ones. After adding validation and limits, re-run the golden set and a normal user session — hardening that breaks legitimate use is a regression, not an improvement.
Q1. Why key rate limits on authenticated user id rather than client IP for an enterprise deployment?
Q2. The groundedness check fails on an answer. The hardened behavior is:
Q3. A user-facing error should contain ___ while the log line contains ___:
Go deeper — curated resources
- Redis-backed rate limiting — The in-process bucket breaks at 2+ replicas. Sketch the Redis INCR/EXPIRE version and note the race it still has — and why that's acceptable here.
- Perimeter suite green; burst test verified with a second unaffected user
- Five failure modes break-tested against the table
- Fresh-clone README run completed without manual fixes
- Golden set still green after hardening; punch list updated with hashes; quiz ≥ 2/3
← Back: Day 45 designed this perimeter, Day 47 taught the token bucket, Day 158 designed the fallback chain, and Day 161 cut the service over — today verified all of it with hostile eyes and the frozen list from Day 175.
Forward →: Tomorrow (Day 177) an inspector re-breaks everything you fixed and re-runs the attack suites; Day 178 ships and documents what survives. The failure-mode table becomes part of the runbook your Demo Day audience can read.
Unlocks: D177 Capstone Quality Gates