Clustering
- Explain the k-means assign/update loop and why it always converges (to something)
- Choose k using the elbow and silhouette methods, and say why neither is an oracle
- Explain when DBSCAN beats k-means and what its two parameters control
- Profile clusters back in original units and give each a name a stakeholder would recognize
- State why scaling is mandatory for distance-based methods
| Spaced-rep warm-up: due cards from Week 11 + Day 50 distance card | 10 min |
| ELI5 + tech read; k-means visualizer (watch centroids walk) | 20 min |
| Guided: k-means on checkable customers + profiling | 37 min |
| Practice: break k-means, call DBSCAN | 20 min |
| Project: segmentation mini-report | 23 min |
| Quiz + flashcards | 10 min |
Builds on: Day 50 — Vectors, norms & distance · Day 69 — Scaling & why it matters · Day 66 — pandas groupby for profiling
You inherit a garage full of a stranger's stuff — no labels, no inventory. So you start making piles: this feels kitchen-ish, that pile is clearly tools, those boxes are holiday decorations. Nobody told you the categories; you invented them by putting similar things near each other and noticing where the gaps fall. Two friends sorting the same garage would produce slightly different piles — and both could be defensible. That is clustering: discovering structure with no answer key.
k-means is a specific sorting strategy: guess k pile-locations, assign every item to its nearest pile, move each pile-marker to the middle of what it collected, and repeat until nothing moves. It is fast and usually sensible, but it has a personality: it wants round piles of similar size, and it will force EVERY item into some pile — the broken umbrella gets jammed into "sporting goods" because k-means has no concept of "this belongs nowhere." DBSCAN is the other personality: it grows piles wherever items are densely packed and honestly labels the stragglers as noise. Today you learn both, and — the part that makes it useful — how to walk back to the piles and name them.
Clustering is the first unsupervised tool customers actually buy: "segment our customers," "group these support tickets," "find anomalous transactions" are recurring FDE briefs, and all are clustering jobs. It is also the analysis layer of modern LLM work: on Day 143 you will cluster embedded production failures to find what KIND of errors dominate — same algorithm, embedding vectors instead of spending columns. And the interview classic "how do you choose k, and what are k-means' failure modes?" is answerable only if you have watched it fail on your own screen, which you will today.
Sorting the garage sale — k-means assigns, then moves the centers
step 1 / 5Eight customers described by one number each (monthly spend). No labels — we want k=2 groups to emerge on their own.
Guided practice
k-means on customers you can check
22 min- Paste the starter. It synthesizes 600 customers from four KNOWN behavioral groups (so you can check the sorting against truth — a luxury real projects never have), scales them, and runs k-means for k = 2…8 printing inertia and silhouette.
- Run it. Find the elbow in the inertia column and the peak in the silhouette column. Do they agree on k = 4?
- Fit the final k=4 model and cross-tabulate found labels vs true groups (
pd.crosstab). Expect near-diagonal — but note any confusion and look at WHICH two true groups blur. - Rerun the whole thing WITHOUT the scaler. Watch silhouette drop and the crosstab smear — write one sentence on which feature's units took over.
- Note cluster label arbitrariness: refit with random_state=1 and watch the same piles get different numbers. Labels are names, not order.
Profile and name the piles
15 min- Build a DataFrame of the ORIGINAL unscaled features plus the k=4 labels: columns spend, visits, tenure, cluster.
df.groupby("cluster").agg(["mean", "count"])— read each cluster's profile against the overall means (df.mean()).- Write one plain-English name and one sentence per cluster, as if labeling a slide for a marketing VP. Compare your names to the true group names in the generator — did the data support the story?
- The habit to keep: profiling happens in ORIGINAL units. Nobody budgets in standard deviations.
On your own
Break k-means, call DBSCAN
20 minGenerate two datasets k-means cannot handle: make_moons(n_samples=400, noise=0.07, random_state=0) and make_blobs(n_samples=[300, 40], centers=[[0, 0], [4, 4]], cluster_std=[1.6, 0.4], random_state=0) (one diffuse blob, one tight one).
Goal: (1) run KMeans(n_clusters=2) on each and describe how it fails (check silhouette AND eyeball a scatter plot or the crosstab if truth is available); (2) run DBSCAN on the moons (start eps=0.2, min_samples=5) and confirm it recovers both crescents plus a few −1 noise points; (3) tabulate DBSCAN's cluster count and noise count for eps in [0.1, 0.2, 0.3, 0.5] — write two sentences on eps sensitivity; (4) one-sentence verdict: for each dataset, which algorithm would you ship?
Hints: k-means slices the moons with a straight frontier because centroid-nearest is a linear boundary between two centroids; DBSCAN follows density. The mixed-density blobs are DBSCAN's OWN weakness — one eps cannot fit both densities.
Segmentation mini-report
Create segments_lab.py + SEGMENTS.md in your practice repo. The script: generates the guided customers (or loads any tabular dataset you prefer), scales, selects k with both elbow and silhouette evidence (printed table), fits final k-means, and emits a per-cluster profile table in original units with population comparison. The report: your chosen k with a two-line justification citing both signals, a named one-sentence profile per cluster, one action a business could take per segment, and a limitations paragraph naming two k-means assumptions your data may violate. This artifact structure — evidence, profile, action, limitations — is exactly the Day 143 failure-clustering deliverable with embeddings swapped in. Commit both files.
Common mistakes & misconceptions
- Clustering unscaled features. Distance-based methods weight features by their units; the biggest-numbered column silently becomes the only feature. Scale first, every time.
- Reading cluster labels as ordered or stable. Label 0 vs 2 is arbitrary naming that changes with random_state; only the groupings mean anything.
- Minimizing inertia to choose k. Inertia monotonically decreases with k — at k = n it hits zero. Use the elbow's bend or silhouette's peak, then apply domain judgment.
- Forgetting k-means assigns EVERYTHING. Outliers get jammed into the nearest pile and drag its centroid; DBSCAN's −1 label, or outlier pruning first, handles them honestly.
- Expecting k-means to find crescents, rings, or elongated groups. Nearest-centroid partitions are convex regions; non-globular shapes need DBSCAN, spectral, or better features.
- Shipping clusters without profiling them. An unlabeled integer per row is not a segmentation — the names, the evidence, and the recommended action are the deliverable.
Q1. You cluster customers on unscaled income (20k–200k) and age (18–80). What happens?
Q2. Why can't you choose k by picking the k with the lowest inertia?
Q3. Two crescent-moon shaped classes. k-means with k=2 fails but DBSCAN succeeds because…
Go deeper — curated resources
- HDBSCAN — density clustering that handles varying density — The mixed-density blobs that broke DBSCAN in practice are HDBSCAN's home turf: it effectively varies eps per region. sklearn ships it as sklearn.cluster.HDBSCAN — rerun your practice data through it.
- k selected with elbow + silhouette agreement; crosstab vs truth examined
- Unscaled rerun performed and the dominating feature named
- DBSCAN recovers the moons; eps-sensitivity table written
- Segmentation report with named, actionable clusters committed
- Quiz ≥ 2/3
← Back: Every distance today is Day 50's Euclidean norm, and the scaling mandate is Day 69's lesson applied to an algorithm that is NOTHING BUT distances. Cluster profiling is Day 66's groupby earning its keep.
Forward →: Tomorrow PCA compresses dimensions so clusters become visible — the standard pairing. On Day 92 embeddings turn text into vectors, and on Day 143 you will cluster embedded production failures with today's exact workflow: scale, cluster, profile, name.
Unlocks: D79 PCA & Dimensionality