Day 16 · The flight recorder

Logging, Config & CLI Ergonomics

You will be able to

Replace print() debugging with the logging module and justify the swap
Configure log levels, formats, and handlers, and toggle verbosity with a flag
Build an argparse CLI with subcommands, typed arguments, and helpful --help text
Load configuration with sane precedence: CLI flag > environment variable > default
Return meaningful exit codes so scripts compose in shell pipelines

Today's ~120 minutes

Spaced-rep warm-up: due flashcards (incl. Day 15 hints)	10 min
ELI5 + tech read: levels, handlers, argparse, exit codes	20 min
Guided: print→logging swap + subcommands & exit codes	40 min
Practice: config precedence function with terminal proof	20 min
Project: the analyzer cockpit	20 min
Quiz + flashcards	10 min

Builds on: Day 5 — Files & errors · Day 6 — The terminal & Linux · Day 14 — Week 2 project — the log analyzer

The analogy

🧸 ELI5⚙ Tech

Airliners carry a flight recorder — the orange "black box" — that continuously writes down altitude, speed, and every switch the crew flips. Nobody reads it during a normal flight. But when something goes wrong over the ocean at 3 a.m., investigators don't have to *reproduce* the incident; they replay the recording. Compare that to a pilot shouting observations out the window: loud, unrecorded, gone.

print() is shouting out the window. The logging module is the flight recorder: every message gets a timestamp, a severity (was this routine cruising data or an engine fire?), and a destination — screen, file, or both. Best of all, the recorder has a dial: in normal operation you record only WARNING and above; when investigating, you turn the dial to DEBUG and the same code suddenly narrates everything, no rewrites needed. Today you also give your programs a proper cockpit: a command-line interface with subcommands and --help, configuration that can come from flags or environment variables, and exit codes — the single number a program leaves behind that tells the rest of the system "landed fine" or "declare emergency."

Why this matters on the job

Every production system you will run — the FastAPI service on Day 42, the RAG capstone from Day 119 — is debugged almost entirely through its logs, because you cannot attach a debugger to a crashed 3 a.m. process. LLM apps raise the stakes: on Day 142 you'll add tracing (the flight recorder's big sibling) to record every prompt, retrieval, and tool call. FDEs live in customer environments where logs are often the ONLY access you get — "send me the log file" is how remote debugging starts. Learn the recorder now, on a small plane.

Guided practice

guided 1

From print to flight recorder

20 min

Create recorder_lab.py from the starter code — it processes fake log lines with print() calls sprinkled in.
Replace every print with the appropriate logger call: routine progress -> logger.debug, notable events -> logger.info, recoverable problems -> logger.warning, failures -> logger.error.
Terminal: run python recorder_lab.py — at the default INFO level the debug chatter is gone.
Add the --verbose flag wiring from the starter (sets level to DEBUG) and run python recorder_lab.py --verbose — the full narration returns without touching the processing code. That is the dial.
Add a FileHandler so everything from DEBUG up also lands in run.log while the console stays at your chosen level. Run once and inspect run.log with cat run.log.

🐍 python — editable, runs in your browser

import argparse
import logging

logger = logging.getLogger("analyzer")

def process(lines):
    parsed = 0
    for i, line in enumerate(lines):
        logger.debug("line %d: %r", i, line)          # routine narration
        parts = line.split(" ")
        if len(parts) < 3:
            logger.warning("skipping malformed line %d: %r", i, line)
            continue
        parsed += 1
    logger.info("parsed %d/%d lines", parsed, len(lines))
    return parsed

if __name__ == "__main__":
    parser = argparse.ArgumentParser(description="Flight-recorder lab")
    parser.add_argument("--verbose", action="store_true", help="show debug output")
    args = parser.parse_args()

level = logging.DEBUG if args.verbose else logging.INFO
    logging.basicConfig(
        level=level,
        format="%(asctime)s %(levelname)-8s %(name)s: %(message)s",
    )
    # file handler: records EVERYTHING regardless of console level
    fh = logging.FileHandler("run.log")
    fh.setLevel(logging.DEBUG)
    fh.setFormatter(logging.Formatter("%(asctime)s %(levelname)s %(message)s"))
    logging.getLogger().addHandler(fh)

process(["10.0.0.1 200 /home", "oops", "10.0.0.2 500 /api"])

Ctrl/⌘+Enter runs · Tab indents · numpy/pandas/sklearn auto-load on import (torch and network calls need a local run)

guided 2

Subcommands and exit codes

20 min

Create cli_lab.py from the starter: an argparse CLI with two subcommands, report and top-ips, each with its own options.
Terminal: run python cli_lab.py --help, then python cli_lab.py top-ips --help — notice argparse wrote both help screens for you.
Run python cli_lab.py report missing.log — it should log an error and exit with code 1. Verify in the terminal with echo $? (PowerShell: echo $LASTEXITCODE) immediately after.
Run it on a real file and verify echo $? prints 0.
Chain it: python cli_lab.py report sample.log && echo "pipeline continues" — the echo only fires on success. This is why exit codes matter.

🐍 python — editable, runs in your browser

import argparse
import logging
import sys
from collections import Counter
from pathlib import Path

logger = logging.getLogger("analyzer")

def cmd_report(args) -> int:
    path = Path(args.logfile)
    if not path.exists():
        logger.error("log file not found: %s", path)
        return 1
    lines = path.read_text().splitlines()
    logger.info("report over %d lines", len(lines))
    print(f"{len(lines)} lines in {path}")
    return 0

def cmd_top_ips(args) -> int:
    path = Path(args.logfile)
    if not path.exists():
        logger.error("log file not found: %s", path)
        return 1
    ips = Counter(line.split(" ")[0] for line in path.read_text().splitlines() if line)
    for ip, n in ips.most_common(args.limit):
        print(f"{ip:15s} {n}")
    return 0

def main() -> int:
    parser = argparse.ArgumentParser(prog="analyzer", description="Log analyzer CLI")
    parser.add_argument("--verbose", action="store_true")
    sub = parser.add_subparsers(dest="command", required=True)

p_report = sub.add_parser("report", help="summary counts")
    p_report.add_argument("logfile")
    p_report.set_defaults(func=cmd_report)

p_top = sub.add_parser("top-ips", help="most frequent client IPs")
    p_top.add_argument("logfile")
    p_top.add_argument("--limit", type=int, default=5)
    p_top.set_defaults(func=cmd_top_ips)

args = parser.parse_args()
    logging.basicConfig(level=logging.DEBUG if args.verbose else logging.WARNING)
    return args.func(args)

if __name__ == "__main__":
    sys.exit(main())

Ctrl/⌘+Enter runs · Tab indents · numpy/pandas/sklearn auto-load on import (torch and network calls need a local run)

On your own

Config precedence, implemented honestly

20 min

Write get_log_level(cli_value) used by your CLI: it must return the level from the CLI flag if given, else from the environment variable ANALYZER_LOG_LEVEL if set, else default to WARNING — and it must reject invalid values with a clear error and exit code 2.

Verify all three tiers in the terminal: (1) no flag, no env var; (2) export ANALYZER_LOG_LEVEL=DEBUG then run (Windows PowerShell: $env:ANALYZER_LOG_LEVEL="DEBUG"); (3) env var set AND --log-level ERROR passed — the flag must win. Prove each with a debug line that only appears at the right tier.

Hints: os.environ.get(name) returns None when unset; logging.getLevelName("DEBUG") maps names to numbers; precedence is just ordered "first non-None wins" logic.

Ship before you stop

Give the analyzer a cockpit

Upgrade your Day 14/15 log analyzer into a proper CLI tool: (1) subcommands report, top-ips, and errors via argparse, each with --limit-style options and real help text; (2) all print-debugging replaced by a module logger with the timestamped format, --verbose dial, and a FileHandler writing analyzer.log; (3) config precedence for the log level (flag > ANALYZER_LOG_LEVEL > default); (4) exit codes: 0 on success, 1 on missing/unreadable input, 2 on bad arguments. Demonstrate the exit codes with a short shell session pasted into your journal. Commit the upgrade — tomorrow this tool gets packaged, and on Day 21 it ships.

Rubric — check what you completed (0/6)

All three subcommands run and --help reads like documentation for eachZero print() calls used for diagnostics (user-facing output only); logger has timestamps and named channel--verbose flips the console to DEBUG without code changes; analyzer.log captures the full recordEnv var ANALYZER_LOG_LEVEL works and is overridden by the CLI flag when both are setecho $? demonstrates 0 / 1 / 2 for the three outcome classesCommitted with a message describing the new interface

Common mistakes & misconceptions

Using print() for diagnostics because "it works." It has no levels, timestamps, destinations, or off-switch — the four things you need the night production breaks. Reserve print for the program's actual output.
Logging at the wrong level: everything at INFO (can never turn the noise down) or everything at ERROR (real alarms drown). Ask: who needs this line, and when?
Calling logging.basicConfig() deep inside library code. Configuration belongs to the application entry point; libraries should only getLogger(__name__) and emit.
Building string messages eagerly: logger.debug("row " + str(huge)) pays the formatting cost even when DEBUG is off. Use lazy %-style: logger.debug("row %s", huge).
Printing "ERROR: ..." but exiting 0. Shells, cron, and CI only see the exit code — a failing script that exits 0 silently corrupts every pipeline built on it.
Committing .env files or secrets in config. Environment-based config exists precisely to keep secrets OUT of the repo; .env goes in .gitignore on Day 20 if it is not there already.

Knowledge check

Q1. Your code is full of logger.debug(...) calls but the level is set to INFO. What happens?

Q2. A script prints "FATAL: could not open file" but ends with sys.exit(0). What does a CI pipeline that ran it conclude?

Q3. With config precedence "flag > env var > default", the env var says DEBUG and the user passes --log-level ERROR. Effective level?

Go deeper — curated resources

docsPython Logging HOWTO (official tutorial) ↗30 min courseMIT Missing Semester — shell tools, env vars & scripting ↗25 min bookAutomate the Boring Stuff — running programs from the command line ↗20 min

If you have a third hour

Structured (JSON) logging — Production services often log one JSON object per line so machines can query logs. Try formatting a record as JSON with the extra= parameter — this becomes standard practice in the capstone (Day 143).

Done means

recorder_lab.py shows the verbosity dial working (INFO vs --verbose) with run.log capturing everything
echo $? verified for success, missing-file, and bad-argument cases
Analyzer upgraded with subcommands, logging, config precedence — committed
Quiz ≥ 2/3

How this connects

← Back: Day 6 gave you the shell where exit codes and env vars live; Day 5 gave you the file handling these subcommands wrap. The analyzer itself is the Day 14 project, freshly cleaned on Day 15.

Forward →: Day 17 packages this CLI as an installable console script. The flight-recorder idea returns at production scale on Day 142 (tracing LLM apps) and the log-mining flywheel on Day 143; environment-based config becomes 12-factor discipline on Day 45.

Unlocks: D17 Packaging & Environments · D39 Operating Systems Essentials · D45 Web Service Architecture · D142 Tracing LLM Applications