ML Code Structure & Pipelines
- Refactor exploratory notebook code into a src-layout project with train/predict entry points
- Build one sklearn Pipeline that owns ALL preprocessing, so serving cannot skew from training
- Drive runs from a config file instead of edited constants
- Persist and reload a fitted pipeline with joblib, with version guards
- Define an inference function contract: raw input in, validated prediction out
| Spaced-rep warm-up: Days 69, 81 due cards | 10 min |
| ELI5 + tech read | 18 min |
| Guided: build the template + inference contract | 40 min |
| Practice: the three tests | 18 min |
| Project: freeze template, README, tag | 24 min |
| Quiz + flashcards | 10 min |
Builds on: Day 69 — ColumnTransformer & leakage discipline · Day 17 — Packaging & project layout · Day 81 — Experiment tracking · Day 18 — pytest fundamentals
A chef invents a brilliant dish at home: a pinch of this, taste, adjust, no measurements written down. Now the restaurant wants it on the menu — 200 plates a night, any cook on shift, identical every time. The home process does not transfer: "a pinch" is not an instruction, "taste and adjust" does not scale, and the chef cannot stand at every stove. The fix is boring and priceless: a written recipe with exact quantities, a mise en place list, and steps any trained cook can execute — the dish, industrialized without being changed.
A notebook is home cooking: perfect for invention (cells re-run out of order, variables tasted and adjusted, state accumulating like a messy counter). But a model that matters gets retrained on fresh data, run by teammates, called by an API at 2 a.m. — restaurant conditions. Today you write the recipe: exploratory code becomes a project with a train script anyone can run, a config file holding every "pinch" as a number, one pipeline object that prepares data identically every time, and a saved model any process can reload and serve. The dish does not change; it becomes repeatable.
The most common production ML bug is training-serving skew: preprocessing done one way in the notebook and a slightly different way in the serving path — scaling with different stats, a category encoded differently — silently degrading every prediction. The single-pipeline-object discipline you learn today is the cure, and interviewers for ML engineering roles probe it directly ("how do you guarantee serving preprocesses like training?"). The template you build today is also compound interest: Day 83 fills it with churn tomorrow, Day 91 refactors the MNIST lab into it, and the Day 119 capstone starts from its skeleton instead of a blank repo.
Guided practice
Build the template — structure first, model second
25 min- Create
ml_template/in your practice repo with the tree from the tech section (touch empty files first;mkdir -p src/churn tests models). - Write
config.yamlwith: data settings (test_size: 0.25, split_seed: 82), model block (learning_rate: 0.1, max_iter: 300), and a features block listing numeric vs categorical column names for the loans data from Day 80 (reuse its generator as data.py's stand-in loader). - Write
features.py:build_pipeline(config)returning the full ColumnTransformer + HistGradientBoosting pipeline. Numeric: passthrough or impute; categorical: OneHotEncoder(handle_unknown="ignore"). - Write
train.pyper the starter shape: load config → load data → split → fit pipeline → evaluate → joblib.dump + metadata JSON → log everything to MLflow via Day 81's conventions. - Run
python -m churn.train --config config.yamlfrom the project root. It should print metrics and leavemodels/model.joblib+models/model_meta.jsonbehind. Change learning_rate in the YAML only, re-run, and confirm a new tracked run appears — an experiment without touching code.
The inference contract + the skew you just made impossible
15 min- Write
predict.py: load the joblib + metadata; warn loudly ifsklearn.__version__differs from the metadata's. Implementpredict_one(raw: dict)— validate required fields are present and numeric fields are numeric, build a one-row DataFrame, return label + probability + model_version (the metadata timestamp or hash). - Test it from the REPL: a valid row, a row with a missing field (should raise a clear error, not a pandas stack trace), and a row with an unseen category (should WORK — that is what handle_unknown="ignore" bought you).
- Now demonstrate the disease you have been vaccinated against: in a scratch script, scale the training data with one StandardScaler, fit a bare LogisticRegression, then "serve" by scaling a test row with a NEW scaler fit on that row alone. Compare predictions vs the pipeline's. Write the moral in one sentence: preprocessing state must travel WITH the model — that is what the single pipeline object does.
On your own
Tests that catch tomorrow's breakage
18 minWrite three pytest tests for the template (Day 18's skills, aimed at ML): (1) a pipeline round-trip test — fit on a small slice, dump, load, and assert identical predictions on 20 rows; (2) a contract test — predict_one raises ValueError on a missing field and succeeds on an unseen category; (3) a training smoke test — train.py's main logic on 200 rows completes and beats the majority-class rate.
Goal: all three pass with pytest tests/; each runs in seconds (small data — these are smoke tests, not benchmarks).
Hints: import functions, don't subprocess the script — refactor main() into testable pieces if needed (that pressure is the point). For (3), compute the majority rate from y itself; asserting beats-dummy is a much better invariant than any hardcoded accuracy.
Freeze the template; write its README
Finish ml_template/ as a reusable starting point: complete tree, working train/predict entry points on the loans stand-in data, the three passing tests, and a README.md documenting: the layout map (one line per file), how to run training and prediction, the config philosophy (code = logic, config = experiments), the inference contract's exact input/output shapes, and a "porting checklist" — the 5 things to change when pointing this template at a new dataset (loader, target, feature lists, config, tests). Tag the commit ml-template-v1. Tomorrow morning, Day 83 starts by copying this directory — the README is the instruction sheet for future-you.
Common mistakes & misconceptions
- Leaving preprocessing logic in the notebook and re-implementing it "the same way" for serving. That duplication IS training-serving skew waiting to happen; one fitted pipeline object, persisted whole, is the fix.
- Hardcoding hyperparameters and paths in the script. Every experiment then edits code, polluting git history and making runs untraceable. Config in, constants out.
- Pickling just the model and scaling "separately" at serve time. The scaler's learned means ARE model state; a fresh scaler on serving data is a different model.
- Loading joblib/pickle files from untrusted sources. Unpickling executes code — treat model files with the same suspicion as executables.
- Ignoring sklearn version drift. A pipeline saved on 1.4 may fail or silently differ on 1.6; save the version in metadata and verify on load.
- Testing ML code with exact-accuracy assertions. Scores wobble with seeds; assert structural invariants instead — beats-dummy, round-trip identity, contract errors.
Q1. Training scaled features with a StandardScaler; the serving path scales incoming rows with statistics computed at serve time. What is this bug called, and what prevents it structurally?
Q2. Why do hyperparameters belong in config.yaml rather than in train.py?
Q3. A good automated test for a trained-model artifact is…
Go deeper — curated resources
- sklearn's set_output and pandas-in/pandas-out pipelines — ColumnTransformer can emit DataFrames (set_output(transform="pandas")), keeping column names through the pipeline — invaluable when debugging what the model actually received. Try it on the template.
- python -m churn.train runs from config and leaves model + metadata behind
- predict_one handles valid, invalid, and unseen-category inputs correctly
- Skew demonstration run and the moral written down
- Three tests pass; template tagged ml-template-v1 with README
- Quiz ≥ 2/3
← Back: Day 69's ColumnTransformer became the "prep" step of one pipeline; Day 17's packaging and Day 18's tests supplied the engineering skeleton; Day 81's tracking plugs into train.py so every run is a lab-notebook entry.
Forward →: Tomorrow the churn project is this template with real stakes — the copy-and-port is milestone zero. Day 91 refactors the MNIST lab into the same shape, and the capstone (Day 119) starts from this skeleton; predict_one becomes a FastAPI endpoint on Day 42's foundations.