Day 123 · A small firm, not one intern

Multi-Agent & Workflow Patterns

You will be able to

State the workflow-vs-agent distinction (Anthropic's framing) and place a given system on that spectrum
Implement routing and evaluator-optimizer as code, and describe chaining, parallelization, and orchestrator-workers precisely
Choose the simplest adequate pattern for three real briefs and defend the choice
Name the costs multi-agent designs add (latency, cost, error compounding, debugging surface) and when they are worth paying

Today's ~120 minutes

Spaced-rep: due cards + the three memory types from memory (D122)	10 min
Read Anthropic's essay + tech section	30 min
Guided: routing + evaluator-optimizer, then paper designs	45 min
Practice: error-compounding budget	18 min
Project: pattern card + triage design	12 min
Quiz + flashcards	5 min

Builds on: Day 120 — Agents I — the loop · Day 121 — Agents II — planning · Day 111 — Tool use & function calling

The analogy

🧸 ELI5⚙ Tech

One brilliant intern can do a lot. But watch a small firm work: the front desk reads each incoming request and sends it to the right specialist (routing). The paralegal drafts, the associate reviews, the draft bounces back until it passes (evaluator-optimizer). Big filings get split across three associates working simultaneously, results merged (parallelization). Some jobs move down a fixed corridor of desks — intake, then research, then drafting, then proofing (chaining). And on the messiest cases, a partner reads the situation, decides which specialists to pull in and in what order, and synthesizes their work (orchestrator-workers).

Here's the part the hype skips: most of these are NOT teams of free-roaming agents. The corridor, the front desk, the review loop — the FIRM designed those paths; the workers just do their step well. That's a workflow: code defines the route, LLMs fill in the steps. A true multi-agent system — several interns each carrying their own to-do list and deciding their own next moves — is the rare, expensive option you save for problems where nobody can draw the corridor in advance. The firm's org chart is a menu, and the skill is ordering the cheapest dish that feeds the problem.

Why this matters on the job

"Should this be one prompt, a pipeline, or agents?" is now THE architecture question in AI engineering interviews and customer engagements — and Anthropic's building-effective-agents essay (today's core reading) became the industry's shared vocabulary for answering it. Getting it wrong is expensive in both directions: mega-prompts that should be three chained calls produce mush; agent swarms that should be a router burn 10x the tokens and are undebuggable. Your Day 126 triage agent is a routing + RAG + approval workflow; your Day 160 system-design interview round will hand you exactly these briefs. FDEs earn trust by prescribing the boring pattern that ships.

Guided practice

guided 1

Build routing and evaluator-optimizer

30 min

Create patterns_lab.py with the starter code: a router (scripted classifier standing in for one cheap LLM call — its prompt is printed) dispatching to three specialized handlers, and an evaluator-optimizer loop improving a draft against explicit criteria.
Run the router on the four requests. Note the shape: classify once, then the SPECIALIZED handler owns the request — each handler's prompt can be short, opinionated, and separately testable. Check the misroute: "my invoice is wrong and the app crashes" is genuinely ambiguous; see which label wins and what the cost of the wrong choice would be.
Run the evaluator-optimizer demo: the drafter produces a reply that violates two explicit criteria (too long, missing citation); the evaluator returns per-criterion pass/fail; the reviser fixes; the loop exits on all-pass. Count the iterations and note the budget cap — this is Day 120's loop discipline applied to quality instead of investigation.
Change MAX_ROUNDS to 1 and observe the honest partial result with its evaluation attached — shipping "best effort + known gaps" beats shipping mystery output.
In your notes, write each of the other three patterns (chaining, parallelization, orchestrator-workers) as three lines of pseudocode. If you can pseudocode it, you own it.

🐍 python — editable, runs in your browser

# patterns_lab.py — routing + evaluator-optimizer, scripted-LLM style.
ROUTER_PROMPT = """Classify the request as exactly one of:
billing | bug | policy. Reply with the label only."""

def route(request):        # stands in for one cheap classifier call
    rl = request.lower()
    scores = {"billing": sum(w in rl for w in ["invoice", "charge", "refund", "price"]),
              "bug":     sum(w in rl for w in ["crash", "error", "broken", "slow"]),
              "policy":  sum(w in rl for w in ["vacation", "policy", "allowed", "expense"])}
    return max(scores, key=scores.get)

def handle_billing(r): return f"[billing handler] check account, invoice tools: {r[:40]}"
def handle_bug(r):     return f"[bug handler] collect version + logs, file ticket: {r[:40]}"
def handle_policy(r):  return f"[policy handler] RAG over handbook (D119): {r[:40]}"
HANDLERS = {"billing": handle_billing, "bug": handle_bug, "policy": handle_policy}

for req in ["I was charged twice this month",
            "the dashboard crashes on export",
            "how many vacation days roll over?",
            "my invoice is wrong and the app crashes"]:   # ambiguous!
    label = route(req)
    print(f"{label:8s} <- {req!r:50s} {HANDLERS[label](req)[:45]}")

# --- evaluator-optimizer: draft -> critique -> revise until pass ---
CRITERIA = {
    "under_50_words": lambda t: len(t.split()) <= 50,
    "has_citation":   lambda t: "[hb-" in t,
    "no_promises":    lambda t: "guarantee" not in t.lower(),
}
def draft(_task):
    return ("We guarantee your refund will be processed. Our policy explains "
            "the full details of the refund process in extensive depth and "
            "many further considerations apply to every possible case, "
            "which we will describe at length below across several sections.")
def revise(text, failures):
    if "no_promises" in failures:
        text = text.replace("We guarantee", "We expect")
    if "has_citation" in failures:
        text += " Per the refund policy [hb-04]."
    if "under_50_words" in failures:
        text = " ".join(text.split()[:38]) + " Per the refund policy [hb-04]."
    return text

MAX_ROUNDS = 3
text = draft("refund reply")
for rnd in range(1, MAX_ROUNDS + 1):
    failures = [name for name, check in CRITERIA.items() if not check(text)]
    print(f"round {rnd}: failures={failures or 'NONE - ship it'}")
    if not failures:
        break
    text = revise(text, failures)     # in prod: evaluator + reviser LLM calls
print("final:", text)

Ctrl/⌘+Enter runs · Tab indents · numpy/pandas/sklearn auto-load on import (torch and network calls need a local run)

guided 2

Three briefs, three designs — on paper

15 min

Design on paper (no code): for each brief, pick the pattern(s), draw the boxes-and-arrows, mark where a human sits, and write the one-line justification. Then compare with the key.

Contract intake: 200 vendor contracts arrive monthly as PDFs; extract 12 fields each into the ERP, flag unusual clauses for legal.
Support inbox: mixed stream of billing/bug/policy emails; billing needs account lookups, bugs need repro details, policy needs handbook RAG.
Competitive research: "Produce a report on how our top 5 competitors price their APIs" — sources unknown in advance, quality bar high.

Key: 1 = chaining with gates (extract → validate schema (D110) → flag clauses → human review for flags) — fixed corridor, zero agency needed. 2 = routing to three specialized handlers, one of which is your capstone RAG; escalation to humans on low router confidence. 3 = orchestrator-workers (decompose per competitor at runtime, parallel research workers, synthesis) + evaluator pass on the report; the one brief where runtime decomposition is genuinely required. If you reached for multi-agent on 1 or 2, reread the costs paragraph.

On your own

The error-compounding budget

18 min

A proposed pipeline chains five LLM stages (intake parse → classify → retrieve+draft → compliance rewrite → format), each independently ~92% reliable on your eval set. (1) Compute expected end-to-end reliability if errors compound independently. (2) You can afford to add programmatic gates (schema/regex checks that catch ~80% of a stage's failures and trigger one retry) after exactly two stages — which two do you protect, and why? (3) Merging the compliance rewrite into the draft stage would remove a hop but grow that prompt's job; name the trade in one sentence. (4) Generalize: write the rule for when to add a stage vs when to merge stages.

Hints: 0.92^5 first. Protect stages whose failures are cheap to DETECT programmatically and expensive to let through; upstream gates protect more downstream work than late ones.

Ship before you stop

The pattern decision card + triage design

Create agent_patterns.md in the practice repo: (1) a six-row table — five workflow patterns + true multi-agent — with mechanism, when-it-wins, failure tax, and a one-line code sketch each; (2) your three paper designs from guided 2, cleaned up; (3) the design for Day 126's support-triage agent, committed to BEFORE building it: routing (ticket class) + RAG handler (capstone retriever as a tool) + evaluator gate on drafted replies (the CRITERIA pattern from today) + human approval on escalations — drawn as boxes and arrows with each pattern labeled; (4) a "multi-agent tax" section: the error-compounding math from practice, plus latency and debugging costs in your own words. This card is your Day 160 system-design cheat sheet.

Rubric — check what you completed (0/6)

All six patterns tabled with mechanism, win condition, and failure taxThree briefs designed with patterns labeled and humans placedDay 126 triage design present with all four components and data flow drawnError-compounding math correct and interpretedSimplest-adequate-pattern principle stated and applied consistentlyCommitted to the practice repo

Common mistakes & misconceptions

Reaching for multi-agent because the problem has parts. Parts want a workflow; free-roaming agency wants unpredictable structure. Most "multi-agent systems" in production are routers and chains.
Chaining without gates. One bad link poisons everything downstream; schema/sanity checks between links catch failures where they are cheapest.
Trusting the router blindly. It is an LLM call with its own error rate — log its decisions, eval it separately (Day 137), and route low-confidence cases to a human or a generalist.
Evaluator-optimizer with vibes criteria. "Make it better" loops forever; explicit, checkable criteria (with the code-checkable subset enforced programmatically) make the loop converge.
Ignoring error compounding. Five 92% stages ≈ 66% end-to-end; every added hop must pay for its reliability tax with a genuine capability gain.
Parallelizing dependent work. Fan-out requires independence; hidden dependencies between branches produce merge conflicts an LLM synthesizer will paper over silently.

Knowledge check

Q1. What distinguishes orchestrator-workers from prompt chaining?

Q2. A five-stage LLM pipeline with 92% per-stage reliability has roughly what end-to-end reliability, and what follows?

Q3. When does evaluator-optimizer actually converge to better output?

Go deeper — curated resources

articleAnthropic — Building Effective Agents (the five patterns; core reading) ↗35 min articleLilian Weng — LLM Powered Autonomous Agents ↗20 min docsLangChain docs — multi-agent & workflow orchestration ↗15 min

If you have a third hour

Handoffs vs agents-as-tools — Two multi-agent wirings: transfer the whole conversation (handoff) vs call a sub-agent like a function and use its return (tool). The second keeps the orchestrator in control and is usually the safer start.

Done means

Router and evaluator-optimizer both run and annotated
Three briefs designed; choices match (or thoughtfully argue with) the key
Compounding math computed and gates placed with reasoning
agent_patterns.md committed including the Day 126 triage design
Quiz ≥ 2/3

How this connects

← Back: The router is Day 111's tool choice one level up; evaluator-optimizer is Day 120's loop aimed at quality; orchestrator-workers is Day 121's planner with delegated executors; parallel fan-out is Day 40's asyncio in costume.

Forward →: Day 126 builds the triage design you just committed to. Day 137 evals the router and trajectories; Day 160's AI system design round hands you these briefs with a whiteboard.

Unlocks: D126 Week 18 Checkpoint: Support-Triage Agent · D160 AI System Design