Probability Fundamentals
- Define sample spaces and events, and compute probabilities by counting or simulating
- Apply the product rule for independent events and the sum rule for mutually exclusive ones
- Explain conditional probability P(A|B) as "probability inside a restricted world"
- Estimate any probability with a Monte Carlo simulation in NumPy and know when to trust it
- Reproduce the birthday-problem result and explain why intuition fails on it
| Spaced-rep warm-up: due flashcards from Week 8 (gradients, matrices) | 10 min |
| ELI5 + tech read: the three rules and the Monte Carlo move | 20 min |
| Guided: Monte Carlo warm-up + birthday problem | 40 min |
| Practice: Monty Hall verdict | 20 min |
| Project: monte_carlo.py toolkit | 20 min |
| Quiz + flashcards | 10 min |
Builds on: Day 50 — Vectors & NumPy arrays · Day 56 — NumPy fluency — linear regression by hand
A weather forecast that says "70% chance of rain" is not saying the sky is 70% full of rain. It is saying: imagine 100 days that look exactly like today on the radar — on about 70 of them it rained. Probability is always that move: instead of asking "what WILL happen?", ask "across all the ways this could play out, what fraction go my way?"
Everything today is that one idea dressed differently. A sample space is the full list of tomorrows. An event is the subset you care about ("it rains"). Conditional probability is checking the forecast after peeking out the window: given that it is already cloudy, the 100 imaginary tomorrows shrink to the cloudy ones, and you recount inside that smaller world. And when the counting gets too hairy to do by hand — which happens fast — you cheat honestly: make the computer live through a million tomorrows and count. That cheat is called Monte Carlo, and it will be your best friend for the whole week.
AI systems are probabilistic top to bottom: an LLM literally outputs a probability distribution over tokens (Day 102), classifiers output P(class|input) (Day 72), and every eval score you will ever report is an estimate from a random sample (Day 139). Engineers who can't reason about probability ship confident nonsense — "the model passed 9/10 tests, it's fine" is a probability error you will learn to catch. The simulation-first habit you build today (simulate, count, then trust the math) is also how working data scientists sanity-check every formula before betting a launch on it.
Guided practice
Monte Carlo warm-up — prove the rules to yourself
20 min- Create
prob_lab.py(or run in the in-browser interpreter) and paste the starter code. - It simulates 200,000 rolls of two dice and estimates P(sum = 7). Check it lands near 1/6 ≈ 0.1667.
- Verify the product rule: estimate P(first die = 6 AND second die = 6) and compare to (1/6)·(1/6) ≈ 0.0278.
- Verify the sum rule with overlap: P(first is 6 OR second is 6). Predict it with the inclusion-exclusion formula (1/6 + 1/6 − 1/36 ≈ 0.3056) before printing.
- Conditional probability: among rolls where the FIRST die shows 6, what fraction have sum ≥ 10? Filter with a boolean mask, then take the mean inside the filtered world. Compare to the by-hand answer (3/6 = 0.5).
- Rerun everything with a different seed. Note how much each estimate wobbles — that wobble is the star of Day 60.
The birthday problem — intuition vs arithmetic
20 min- Before any code: write down your gut guess for "how many people in a room before a shared birthday is a coin flip (50%)?" Most people guess 100+.
- Paste the starter code. It simulates rooms of size k and estimates the collision probability for k = 5..60.
- Find the smallest k where the estimate crosses 0.5. (It is 23.)
- Now compute the exact answer with the complement trick — the product loop in the code — and confirm simulation and formula agree to ~2 decimal places.
- Write two sentences in your notes: why does intuition fail? (Hint: you compare 23 to 365, but the number of PAIRS in a room of 23 is 253 — collisions scale with pairs.)
- AI tie-in: replace 365 with 4096 "hash buckets" and find the 50% collision point. This is exactly how you reason about ID collisions and cache-key clashes in real systems.
On your own
The Monty Hall verdict
20 minSettle the most argued-about probability puzzle in history with a simulation. Setup: a prize hides behind one of three doors; you pick one; the host (who knows where the prize is) opens a different door that is empty; you may stick or switch.
Your goal: simulate 100,000 games under the "always stick" strategy and 100,000 under "always switch", and print both win rates. Then write a 3-sentence explanation of the result using conditional probability language ("given that the host opened…").
Constraints: model the host correctly — the host never opens your door and never reveals the prize. That constraint is the entire puzzle.
Hints (only if stuck): the switch strategy wins exactly when your first pick was wrong. What is the probability your first pick was wrong?
Your Monte Carlo toolkit
Create monte_carlo.py in your practice repo: a small reusable module with a function estimate(event_fn, n_trials, seed) that runs a vectorized simulation and returns the estimated probability, plus a demo section answering three questions of YOUR choosing by simulation (e.g. "probability a 5-request retry chain with 80% per-try success eventually succeeds", "probability two of my 10 microservices fail in the same hour if each fails 1% of hours"). For each, state the assumption you made (independence? equal likelihood?) in a comment. Commit it — Days 60, 61, and 63 import ideas (and possibly code) from this file.
Common mistakes & misconceptions
- Adding probabilities of events that can co-occur. P(A or B) needs the −P(A and B) correction unless the events are mutually exclusive.
- Multiplying probabilities of events that are not independent. Correlated failures (same server, same outage) make P(both fail) far higher than the product.
- Reading P(A|B) as P(B|A). "P(sum≥10 | first die is 6)" and "P(first die is 6 | sum≥10)" are different numbers — the confusion Bayes fixes on Day 59.
- Trusting a Monte Carlo estimate without checking its noise. 1,000 trials of a 1%-probability event sees ~10 hits; the estimate can easily be off by 50%. Scale trials to the rarity of the event.
- Forgetting to seed the generator, then being unable to reproduce a "weird" result. Always pass an explicit seed; rerun with a second seed to check stability.
- Computing "at least one" head-on with a giant case analysis instead of 1 − P(none). The complement trick is almost always the shorter road.
Q1. You roll two fair dice. What is P(sum ≥ 10 GIVEN the first die shows 6)?
Q2. Why does "P(at least one shared birthday among 23 people) ≈ 50%" feel too high to most people?
Q3. A Monte Carlo run with 1,000 trials estimates a probability as 0.012. What should you do before reporting it?
Go deeper — curated resources
- Stat 110 Lecture 1–2 — sample spaces and counting done rigorously ↗ — Blitzstein's counting arguments (naive definition, multiplication rule) put today's intuitions on formal footing.
- All guided simulations run and match theory to ~2 decimals
- Monty Hall simulation shows ~1/3 stick vs ~2/3 switch, with a written conditional-probability explanation
- monte_carlo.py committed with three original questions answered
- Quiz ≥ 2/3 (retake after revisiting if lower)
← Back: The vectorized boolean-mask tricks come straight from Day 50's NumPy work — a mean over a boolean array is a dot product with a ones vector divided by n. Day 56 gave you the NumPy fluency this week leans on daily.
Forward →: Day 58 gives the outcomes shapes (distributions); Day 59 turns conditional probability into Bayes' rule; Day 60 explains exactly how fast Monte Carlo converges. On Day 102 you'll watch an LLM sample from a probability distribution over tokens, and on Day 139 every eval score becomes a probability estimate with error bars.
Unlocks: D58 Random Variables & Distributions · D59 Bayes' Rule · D60 Statistics I — Sampling & Confidence · D62 Entropy, Cross-Entropy & KL