Random Variables & Distributions
- Distinguish discrete from continuous random variables and read their distributions
- Recognize Bernoulli, binomial, uniform, and normal distributions and say what process generates each
- Compute expectation and variance by simulation and by formula, and explain what each summarizes
- Demonstrate the Central Limit Theorem by simulation and state what it does and does not say
- Explain why production latencies are heavy-tailed and why that makes means misleading
| Spaced-rep warm-up: due cards + Day 57 rules recall | 10 min |
| ELI5 + tech read, study the dist-shapes visualizer | 20 min |
| Guided: four shapes + CLT demonstration | 38 min |
| Practice: the tail wags the SLA | 20 min |
| Project: distribution field guide | 22 min |
| Quiz + flashcards | 10 min |
Builds on: Day 57 โ Probability fundamentals & Monte Carlo ยท Day 50 โ NumPy arrays & vectorized math
Flip a coin once and the outcome is chaos โ you know nothing. Flip it a thousand times and something eerie happens: the chaos develops a *shape*. Roughly 500 heads, a predictable bell of wobble around it, and you can say exactly how surprised to be by 530. Randomness is unpredictable one draw at a time but astonishingly disciplined in bulk, and a distribution is the name of that discipline โ a portrait of where the outcomes pile up.
A random variable is just "a number the world hands you at random": the number of failed requests today, the milliseconds a query took. Its distribution is the histogram you would see given infinite repeats. A few shapes cover most of reality: the coin flip (Bernoulli), coins-in-bulk (binomial), the flat "anything goes" (uniform), and the bell (normal). Two numbers summarize any shape: expectation โ where the pile balances โ and variance โ how wide it sprawls. The deepest magic is the Central Limit Theorem: average enough draws from almost ANY shape and the average itself turns bell-shaped. That is why the bell curve is everywhere: it is what averaging does to the world.
Distributions are the vocabulary of every measurement you will make as an AI engineer. An eval pass rate is a binomial; response latency is log-normal-ish with a brutal tail, which is why Day 155 talks p95/p99 instead of averages; the CLT is the reason confidence intervals (Day 60) and significance tests (Day 61, Day 139) work at all. Engineers who only know "the average" get burned in production weekly: a service with a fine mean latency and an ugly p99 is a service your biggest customer experiences as broken.
The shape of chance โ three ways latency can be random
step 1 / 6Uniform: every value in a range is equally likely โ a fair spinner. Flat top, hard edges. Rare in nature, common in simulations.
Guided practice
Meet the four shapes โ simulate, summarize, compare to theory
20 min- Paste the starter code. It draws 100,000 samples each from Bernoulli(0.3), Binomial(40, 0.85), Uniform(0, 1), and Normal(200, 25).
- For each, compare the simulated mean and standard deviation against the theory printed alongside. They should agree to ~2 significant figures.
- The Binomial(40, 0.85) line is your first eval-score distribution: "40 golden-set cases, each passing with probability 0.85". Look at its spread โ a run of 34/40 (85%) and a run of 31/40 (77.5%) are both completely ordinary. Sit with that.
- Change the binomial to n=400 cases. Watch the standard deviation of the pass RATE shrink by โ10 โ 3.2ร. This is why bigger golden sets buy sharper conclusions (Day 140 builds a 40+ case set for exactly this reason).
- For the normal draws, verify the 68/95/99.7 rule with boolean masks.
Watch the Central Limit Theorem happen
18 min- The starter draws from a deliberately ugly distribution: an 80/20 mixture (80% fast draws near 1, 20% slow draws near 10) โ bimodal, skewed, nothing like a bell.
- Compute means of samples of size n = 1, 5, 30, 200 (20,000 sample-means each) and print each collection's mean, standard deviation, and a crude text histogram.
- Observe two things as n grows: the spread of the means shrinks like 1/โn, and the histogram of MEANS morphs into a bell even though the raw data never does.
- Verify the shrink quantitatively: sd(means at n=200) should be close to sd(raw)/โ200.
- Write one sentence in your notes: "The CLT is about the distribution of the ______, not the distribution of the ______." Fill the blanks.
On your own
The tail wags the SLA
20 minSimulate 50,000 request latencies as log-normal: rng.lognormal(mean=4.0, sigma=0.8) milliseconds. Your goals: (1) compute mean, median (p50), p95, and p99; (2) explain in two sentences why the mean is higher than the median and which number you would put in a customer-facing SLA; (3) now contaminate the sample โ replace 1% of requests with timeouts at 30,000 ms โ and report how much the mean vs the p50 vs the p99 moved. (4) Write the one-line takeaway an SRE would tattoo on their arm.
Constraints: no plotting needed โ quantiles via np.percentile tell the story. Hints: log-normal means "the LOG of the latency is normal" โ multiplicative effects (retries, queue depth, payload size) make this shape; the median barely notices outliers because it only counts ranks, not magnitudes.
A distribution field guide for production numbers
Create distributions_notes.md in your practice repo. For each of five quantities โ (a) one eval case passing, (b) passes out of n eval cases, (c) a random seed's uniform draw, (d) an average of many small independent effects, (e) request latency โ name the distribution family, the parameters you'd need to estimate, the honest summary statistic (mean? median? p99?), and one sentence on why. Then add a short simulation appendix (distributions_lab.py) reproducing today's CLT demo at n = 30 and the latency percentile table. This file is your cheat sheet when Day 139 asks "what distribution is a pass rate?"
Common mistakes & misconceptions
- Reporting the mean of a heavy-tailed quantity. Latency and cost distributions have outliers that drag the mean above what most users experience โ quote p50/p95/p99.
- Believing the CLT makes your DATA normal. It makes the SAMPLE MEAN approximately normal; the raw data keeps its shape forever.
- Treating a 3-point drop in a 40-case eval as signal. Binomial(40, 0.85) has sd โ 2.3 cases โ swings of 2โ3 cases are pure noise. Day 139 formalizes the fix.
- Confusing standard deviation (spread of the data) with standard error (spread of the mean, = sd/โn). The first stays put as n grows; the second shrinks.
- Asking P(X = exact value) for a continuous variable. It is zero; only ranges (areas under the density) carry probability.
- Assuming independence to use binomial math when trials are correlated โ eval cases sharing a template, requests sharing a bad server. Correlation widens the real spread.
Q1. Your model truly passes 85% of cases. On a 40-case golden set, one run scores 31/40 (77.5%). What is the best interpretation?
Q2. The Central Limit Theorem says that as n growsโฆ
Q3. For request latency, the mean is 95 ms but the median is 55 ms. Why, and which belongs in the SLA conversation?
Go deeper โ curated resources
- Poisson processes โ counts of rare events per interval โ Requests per second, errors per day: Poisson(ฮป) with mean = variance = ฮป. A 10-minute read that instantly upgrades your on-call reasoning.
- Simulated means/sds match theory for all four distributions
- CLT demo shows the 1/โn shrink and the bell shape at n=200
- Latency exercise: percentile table computed and SLA takeaway written
- Field guide committed; quiz โฅ 2/3
โ Back: Every simulation today is Day 57's Monte Carlo move at scale, and the vectorized (trials ร n) arrays are Day 51's matrix thinking paying rent.
Forward โ: Day 60 turns "sd of the sample mean" into standard errors and confidence intervals; Day 61 tests whether two shapes differ. Binomial eval noise returns as the villain of Day 139, and latency percentiles headline Day 155's serving work.
Unlocks: D59 Bayes' Rule ยท D60 Statistics I โ Sampling & Confidence ยท D61 Statistics II โ Hypothesis Tests & A/B ยท D67 Exploratory Data Analysis