Day 45 · The restaurant's back of house

Web Service Architecture

You will be able to

Restructure a FastAPI app into routes, services, and repositories with clean boundaries
Explain dependency injection and use FastAPI Depends for auth and DB access
Add middleware for request IDs and timing, and consistent problem-detail errors
Implement simple rate limiting and explain where it belongs in the stack
Apply 12-factor config so the same code runs in dev and prod

Today's ~120 minutes

Spaced-rep warm-up: due flashcards (auth + networking)	10 min
ELI5 + tech read: back-of-house layering	15 min
Guided: carve the monolith + middleware	40 min
Practice: token-bucket rate limiter	20 min
Project: harden the Day 42 service	25 min
Quiz + flashcards	10 min

Builds on: Day 42 — The API + DB mini-service · Day 44 — AuthN & AuthZ · Day 16 — Logging, config & CLI ergonomics

The analogy

🧸 ELI5⚙ Tech

Walk into a good restaurant's back of house and you will not find one heroic cook doing everything. The waiter (routes) takes orders and speaks "menu" to customers — they never touch a pan. The kitchen (service layer) knows the recipes — the business logic — but never talks to diners. The pantry crew (repository layer) fetches ingredients from storage and is the only team that knows which shelf holds what. Between the door and the waiter stand the host and bouncer (middleware): greeting everyone the same way, checking reservations, throwing out troublemakers — before any order is taken.

When a restaurant mixes these jobs — the waiter cooking, the chef running to the cellar — small changes cause chaos: change the menu wording and somehow the soup breaks. Layered services exist for the same reason: each layer speaks only to its neighbors, so you can swap the pantry (SQLite → Postgres), rewrite a recipe (business rule), or retrain the waiters (API version) without the other layers noticing. Today you rebuild your Day 42 service this way and give it a bouncer.

Why this matters on the job

Every LLM backend you will build — the capstone included — is a FastAPI-shaped service: routes receiving chat requests, a service layer orchestrating retrieval and model calls, repositories wrapping the vector store and DB. Teams reject FDE prototype code that tangles these layers because they cannot extend it. Middleware is where production non-negotiables live: request IDs that let you find one user's failing request in a million log lines (the tracing habit Day 142 formalizes), rate limits that stop one customer's retry storm from bankrupting your token budget. This architecture is the difference between a demo and something a customer can run.

Guided practice

guided 1

Carve the monolith into layers

25 min

Take your Day 42 notes service (or the reference shape in the starter) and restructure it.

Create the package layout: app/routes/notes.py, app/services/notes.py, app/repos/notes.py, app/deps.py, app/main.py.
Move ALL SQL into the repo class. Its methods speak domain language (list_for_owner, add, delete), not SQL.
Move rules into the service: owner checks, title validation. The service raises domain exceptions (NotFound, Forbidden) — it has no idea HTTP exists.
Routes become thin: parse → call service → return. Wire the repo and current user via Depends.
Prove the payoff: write one service test using an in-memory fake repo — no HTTP, no database, runs in milliseconds.

🐍 python — editable, runs in your browser

# app/repos/notes.py — the ONLY file that knows SQL
import sqlite3

class NotesRepo:
    def __init__(self, conn: sqlite3.Connection):
        self.conn = conn

def list_for_owner(self, owner_id: int) -> list[dict]:
        rows = self.conn.execute(
            "SELECT id, title, body FROM notes WHERE owner_id = ?",
            (owner_id,)).fetchall()
        return [dict(r) for r in rows]

def add(self, owner_id: int, title: str, body: str) -> int:
        cur = self.conn.execute(
            "INSERT INTO notes (owner_id, title, body) VALUES (?, ?, ?)",
            (owner_id, title, body))
        self.conn.commit()
        return cur.lastrowid

# app/services/notes.py — rules only, no HTTP, no SQL
class NotFound(Exception): pass
class Forbidden(Exception): pass

class NotesService:
    def __init__(self, repo):
        self.repo = repo

def create(self, user: dict, title: str, body: str) -> int:
        if not title.strip():
            raise ValueError("title must not be empty")
        return self.repo.add(user["id"], title.strip(), body)

# tests/test_service.py — the layering payoff: no DB needed
class FakeRepo:
    def __init__(self): self.rows = []
    def add(self, owner_id, title, body):
        self.rows.append((owner_id, title, body)); return len(self.rows)

def test_create_strips_title():
    svc = NotesService(FakeRepo())
    assert svc.create({"id": 1}, "  hi  ", "b") == 1

Ctrl/⌘+Enter runs · Tab indents · numpy/pandas/sklearn auto-load on import (torch and network calls need a local run)

guided 2

Middleware: request IDs, timing, honest errors

15 min

Add the middleware from the starter to your app: every request gets a short request ID, is timed, and logged as one structured line.
Add the exception handlers: domain NotFound → 404 problem-details, Forbidden → 403, ValueError → 422. Confirm all errors now share one JSON shape.
Hit a route with the docs UI, find your request ID in the logs, and confirm the same ID came back in the X-Request-ID response header — this is how you will correlate a user's bug report to a log line.
Trigger each error deliberately and screenshot-or-paste the consistent bodies into your notes.

🐍 python — editable, runs in your browser

Ctrl/⌘+Enter runs · Tab indents · numpy/pandas/sklearn auto-load on import (torch and network calls need a local run)

On your own

Build a token-bucket rate limiter

20 min

Implement rate limiting for your service with no library: a token bucket per API key (capacity 10, refill 1 token/second), enforced in middleware or a dependency, returning 429 with a Retry-After header when empty.

Constraints: pure stdlib; buckets in a module-level dict (note in a comment why this breaks with multiple instances — Day 46 explains, Day 47 fixes it with Redis); a unit test that fires 11 instant requests and asserts the 11th gets 429; a second test that sleeps past a refill and asserts recovery.

Hints (only if stuck): store (tokens, last_refill_time) per key; on each request, first credit elapsed_time × rate up to capacity, then try to spend 1.

Ship before you stop

Harden the Day 42 service

Ship v1 of the notes service with today's architecture: layered packages (routes/services/repos), API-key auth using Day 44's auth_lib wired through Depends (keys stored hashed; a POST /v1/keys bootstrap route returns a key once), request-ID + timing middleware, problem-details errors, your token-bucket limiter, /v1 path versioning, and all config (DB path, rate limit, admin bootstrap secret) from env vars with sane defaults. Tests: service-layer tests on a fake repo plus TestClient tests proving 401 without a key, 429 over the limit, and consistent error bodies. This service is your template for every API in the program — the capstone (Day 119) starts from this shape.

Rubric — check what you completed (0/6)

SQL exists only in the repo layer; grep proves itProtected routes return 401 without a valid key and 200 with oneRate limiter returns 429 with Retry-After on the 11th instant requestAll error responses share the problem-details JSON shapeService tests run with a fake repo and no HTTP; full suite passesNo secret or environment-specific value is hard-coded (checked against 12-factor)

Common mistakes & misconceptions

Putting business logic in routes "because it is short." It never stays short, and logic in routes can only be tested through HTTP. Routes translate; services decide.
Letting SQL leak into services. The repo boundary is what makes storage swappable and service tests fast; one stray query breaks both.
Skipping dependency injection and importing globals directly. Hard-wired dependencies make tests require real databases and real keys; Depends + overrides keep tests honest and fast.
Inventing a new error JSON per endpoint. Clients end up parsing four shapes; one problem-details handler ends it.
Rate limiting inside handlers after work is done. Limit at the edge (middleware/dependency) BEFORE expensive work — the point is protecting the kitchen, not scolding afterwards.
Deferring versioning "until v2." Once clients exist, unversioned paths are permanent contracts. /v1 costs nothing today and saves a migration later.

Knowledge check

Q1. In a layered service, which statement about dependencies is correct?

Q2. What is the main testing benefit of injecting the repository into the service rather than constructing it inside?

Q3. A token bucket has capacity 10 and refills 1 token/sec. A client idle for an hour then bursts 15 instant requests. What happens?

Go deeper — curated resources

docsFastAPI — Bigger Applications (APIRouter structure) ↗25 min docsThe Twelve-Factor App ↗25 min docsFastAPI — Security & Depends ↗20 min repoSystem Design Primer — application layer ↗15 min

If you have a third hour

RFC 9457 — Problem Details for HTTP APIs — The standard behind the error shape you built: media type application/problem+json and the meaning of each field.

Done means

Layered restructure done; service test passes with fake repo
Request IDs visible in logs and response headers
Rate limiter passes both unit tests (11th request 429; refill recovers)
Hardened service committed: auth, versioning, env config, tests green
Quiz ≥ 2/3

How this connects

← Back: This is Day 42's service grown up: Day 44's auth_lib guards the door, Day 16's logging and env-config habits fill the middleware and settings, and the repo layer isolates the SQL you learned on Days 36–38.

Forward →: Day 46 explains why your limiter's in-memory dict breaks the moment you run two instances — statelessness — and Day 47 moves that state to a shared cache. The capstone service (Day 119 onward) starts from today's template, and Day 142's tracing upgrades your request IDs into full spans.

Unlocks: D46 Distributed Systems Fundamentals · D47 Caching & Queues · D49 Week 7 Checkpoint: Design & Build a URL Shortener · D169 Enterprise Integration