Day 118 · Beyond one shelf

Advanced RAG Patterns

You will be able to

Decompose multi-hop questions into dependent sub-queries and execute them sequentially
Sketch when graph-RAG, RAPTOR-style summary trees, and agentic retrieval each earn their complexity
Route between structured (SQL) and unstructured (vector) sources for a mixed question
Assemble the week into a failure-mode taxonomy and a debugging flowchart you will use on the capstone

Today's ~120 minutes

Spaced-rep: due cards + bi- vs cross-encoder from memory (D117)	10 min
ELI5 + tech read	25 min
Guided: multi-hop + routing labs	40 min
Practice: prescription drill	18 min
Project: debugging flowchart	22 min
Quiz + flashcards	5 min

Builds on: Day 117 — Reranking & query transforms · Day 113 — RAG architecture · Day 111 — Tool use & function calling

The analogy

🧸 ELI5⚙ Tech

Some questions can't be answered from one shelf. "Did the author of our expense policy also write the security policy?" needs three trips: find who wrote the expense policy, find who wrote the security policy, compare. A single search for the whole question retrieves a muddle — no one page contains the answer, because the answer doesn't LIVE anywhere; it has to be assembled.

The advanced patterns are all ways of making multiple, smarter trips. Multi-hop decomposition breaks the question into steps where each search uses the previous answer. A summary tree (RAPTOR) adds "shelf summaries" and "floor summaries" to the library so zoomed-out questions ("what are the big themes?") have something zoomed-out to retrieve. Graph-RAG draws a map of which people, teams, and documents connect to each other, so "everything related to X" follows edges instead of similarity. And agentic retrieval hires the intern from next week early: let the model itself decide what to search next, look at what came back, and search again — an open-book exam where the student may return to the shelves as often as needed.

Why this matters on the job

One-shot retrieval tops out fast on real corpora: enterprise users ask comparison questions, aggregate questions ("how many customers mentioned pricing?"), and questions whose answers span three systems — the exact queries that make naive RAG look broken in a pilot. Knowing which pattern fixes which failure — and that each adds latency, cost, and failure surface — is the difference between an engineer and a framework-user. The debugging flowchart you build today is a genuine FDE artifact: when a customer says "it answered wrong," you'll walk the flowchart live instead of shrugging. It also completes the mental model your capstone and Day 136's evals depend on.

Guided practice

guided 1

Multi-hop over a corpus where one-shot fails

25 min

Create multihop_lab.py with the starter code: a corpus with document authorship facts and a hand-written two-hop plan for the question "Did the author of the expense policy also write the security policy?"
Run the one-shot baseline: the full question retrieves a muddle (verify: no single chunk contains the answer).
Run the multi-hop version: hop 1 retrieves the expense policy's author chunk; a tiny extractor pulls the name; hop 2's query is BUILT from that name; the synthesizer compares. Note the dependency — hop 2 could not have been written in advance.
Poison hop 1: change the extractor to return the wrong name and watch hop 2 confidently retrieve irrelevant evidence. Write one sentence on why intermediate verification matters.
In production the plan comes from an LLM ("split this question into sequential sub-queries; later queries may reference earlier answers as ⟨answer1⟩") — the printed PLAN_PROMPT shows exactly that prompt.

🐍 python — editable, runs in your browser

import math, re
from collections import Counter

CORPUS = {
    "meta-1": "The expense policy was written by Dana Okafor of the Finance team, last revised March 2026.",
    "meta-2": "The security policy was written by Priya Natarajan of the Security team.",
    "meta-3": "The vacation policy was written by Dana Okafor and approved by HR.",
    "hb-1": "Purchases over 500 USD require manager pre-approval.",
    "sec-1": "Laptops must use full-disk encryption; report loss within 4 hours.",
    "hr-1": "Employees accrue 1.5 vacation days per month.",
}
def vec(t): return Counter(re.findall(r"[a-z0-9-]+", t.lower()))
def cosine(a, b):
    dot = sum(a[w] * b[w] for w in a if w in b)
    na, nb = math.sqrt(sum(v*v for v in a.values())), math.sqrt(sum(v*v for v in b.values()))
    return dot / (na * nb) if na and nb else 0.0
V = {i: vec(c) for i, c in CORPUS.items()}
def retrieve(q, k=2):
    qv = vec(q)
    return sorted(CORPUS, key=lambda i: -cosine(qv, V[i]))[:k]

QUESTION = "Did the author of the expense policy also write the security policy?"

print("one-shot:", retrieve(QUESTION, k=3), "  <- muddle: answer spans two chunks")

# --- multi-hop: dependent sub-queries ---
hop1_q = "who wrote the expense policy"
hop1_hits = retrieve(hop1_q, k=1)
hop1_text = CORPUS[hop1_hits[0]]
m = re.search(r"written by ([A-Z][a-z]+ [A-Z][a-z]+)", hop1_text)
author = m.group(1) if m else "UNKNOWN"          # in prod: an LLM extraction call
print(f"hop1: {hop1_hits} -> author = {author}")

hop2_q = f"who wrote the security policy {author}"   # query BUILT from hop1
hop2_hits = retrieve(hop2_q, k=1)
hop2_text = CORPUS[hop2_hits[0]]
print(f"hop2: {hop2_hits} -> {hop2_text}")

same = author in hop2_text
print(f"synthesis: expense author is {author}; security policy chunk "
      f"{'names' if same else 'does not name'} them -> answer: {'Yes' if same else 'No'}")

PLAN_PROMPT = """Split the question into sequential sub-queries. Later
sub-queries may reference earlier answers as <answer1>, <answer2>.
Question: Did the author of the expense policy also write the security policy?
Sub-queries:
1. Who wrote the expense policy?
2. Did <answer1> write the security policy?"""
print("\n--- the prompt that generates such plans ---\n" + PLAN_PROMPT)

Ctrl/⌘+Enter runs · Tab indents · numpy/pandas/sklearn auto-load on import (torch and network calls need a local run)

guided 2

Route the question: SQL, vectors, or both

15 min

Append the starter code: a mini "warehouse" (a list of ticket dicts), the vector corpus from exercise 1, and a keyword-based router standing in for an LLM classification call (the real router prompt is printed).
Run the three test questions: an aggregate ("how many tickets mention refunds"), a policy lookup ("what needs pre-approval"), and a mixed one ("are refund complaints against our stated policy?").
Verify the aggregate goes to the SQL-ish path and returns an exact count — then force it through vector retrieval instead and see why that's malpractice (it returns 2 example tickets, not a count).
For the mixed question, confirm both paths run and note what the synthesis step would need to cite: a number from one source, a quote from the other.

🐍 python — editable, runs in your browser

TICKETS = [
    {"id": 1, "text": "I want a refund for the annual plan", "quarter": "Q2"},
    {"id": 2, "text": "dashboard is slow", "quarter": "Q2"},
    {"id": 3, "text": "refund please, feature missing", "quarter": "Q2"},
    {"id": 4, "text": "love the product!", "quarter": "Q1"},
    {"id": 5, "text": "requesting refund per policy", "quarter": "Q1"},
]

def sql_count_mentions(term, quarter=None):
    rows = [t for t in TICKETS if term in t["text"].lower()
            and (quarter is None or t["quarter"] == quarter)]
    return len(rows)

def route(question):
    ql = question.lower()
    aggregate = any(w in ql for w in ["how many", "count", "average", "per quarter"])
    policyish = any(w in ql for w in ["policy", "allowed", "approval", "rule"])
    if aggregate and policyish: return "both"
    if aggregate: return "sql"
    return "vector"

ROUTER_PROMPT = """Classify the question as one of: sql (aggregate/count/trend
over records), vector (policy/docs lookup), both. Reply with the label only."""

for q in ["how many tickets mention refunds in Q2?",
          "what purchases need pre-approval?",
          "how many refund requests did we get, and is refusing them against policy?"]:
    r = route(q)
    print(f"\nQ: {q}\n  route -> {r}")
    if r in ("sql", "both"):
        print("  sql path: refund mentions =", sql_count_mentions("refund"))
    if r in ("vector", "both"):
        print("  vector path:", retrieve(q, k=1))
print("\n(real router = one small LLM call with:\n" + ROUTER_PROMPT + ")")

Ctrl/⌘+Enter runs · Tab indents · numpy/pandas/sklearn auto-load on import (torch and network calls need a local run)

On your own

Pattern-matching drill: prescribe, don't implement

18 min

For each symptom below, prescribe the cheapest adequate fix from this week's toolbox (chunking / hybrid / rerank / rewrite / HyDE / decomposition / RAPTOR / graph-RAG / routing / freshness), plus one sentence of reasoning. Then rank your prescriptions by implementation cost.

"Summarize our Q2 customer themes" retrieves five random specific tickets.
"Compare the old and new expense policies" answers from only one of them.
Follow-up questions in chat retrieve nonsense.
"Who has worked with vendor Acme across any team?" misses connections spread over many docs.
Answers cite a policy retired two months ago.
"SKU-9981 warranty length" returns unrelated warranty prose.

No peeking until done. Reasonable answers: 1 RAPTOR-style summaries (no zoomed-out chunks exist); 2 decomposition (two targeted retrievals + compare); 3 query rewriting; 4 graph-RAG (relationship question); 5 freshness/incremental indexing + tombstones; 6 hybrid (exact identifier).

Ship before you stop

The RAG debugging flowchart

Create rag_debugging.md — the artifact you will physically use during the capstone and can show in interviews. Part 1: the failure taxonomy as a table (symptom → stage → root cause → fix → which day taught it) covering at least 9 failures from this week. Part 2: a flowchart (Mermaid or ASCII) starting from "answer is wrong" that branches on OBSERVABLE checks in debugging order: was the doc ingested? → is the fact intact in one chunk? → is that chunk in stage-one top-20? → in reranked top-3? → did the prompt contain it? → did the model use it faithfully? Each leaf names the fix. Part 3: three symptoms where the fix is an ADVANCED pattern (decomposition, routing, summaries), so you don't over-prescribe them for stage-level bugs. Commit it.

Rubric — check what you completed (0/6)

Taxonomy table has ≥ 9 symptom rows, each mapped to stage, fix, and source dayFlowchart branches only on checks you could actually run against logs/indexCheck order goes upstream-to-downstream (ingest before generation)Advanced patterns appear as leaves only where one-shot retrieval is structurally insufficientA teammate could triage a failure using only this fileCommitted to the practice repo

Common mistakes & misconceptions

Prescribing agentic/multi-hop retrieval for stage-level bugs. If the chunk was torn or the identifier unsmellable, fix chunking or add BM25 — advanced patterns multiply cost and failure surface.
Letting hop-2 run on an unverified hop-1 answer. Dependent hops compound errors; verify intermediate extractions (or at least carry confidence forward) when hops chain.
Answering aggregate questions from a vector index. "How many…" needs SQL over records; similarity search returns examples, not counts — route, don't retrieve.
Building RAPTOR trees or knowledge graphs before query logs demand them. Both are heavy at ingest and painful on updates; earn them with observed thematic/relationship queries.
Forgetting deletions. Removing a doc from the source without tombstoning its chunks means the retired policy keeps answering — freshness includes forgetting.
Treating the debugging flowchart as documentation. It is a runbook: walk it top-down on every capstone failure, and update it when a new failure class appears.

Knowledge check

Q1. "Which of our policies were written by the same person?" fails with one-shot retrieval because…

Q2. A user asks "how many tickets mentioned refunds last quarter?" The correct architecture move is…

Q3. What problem do RAPTOR-style summary trees solve?

Go deeper — curated resources

paperRAPTOR paper — recursive abstractive retrieval (Sarthi et al.) ↗30 min docsLlamaIndex — advanced retrieval (routing, sub-questions) ↗25 min articleAnthropic — building effective agents (augmented LLM & routing sections) ↗20 min

If you have a third hour

GraphRAG (Microsoft) — community summaries — Combines both structure patterns: build an entity graph, detect communities, summarize each community — graph structure plus RAPTOR-style altitude in one system.

Done means

Multi-hop lab run, including the poisoned-hop experiment
Router demonstrated on all three question types, with the malpractice case shown
All six prescriptions reasoned and cost-ranked
rag_debugging.md committed with taxonomy + flowchart
Quiz ≥ 2/3

How this connects

← Back: This day is Week 17's synthesis: Day 113's stages became Day 114–117's fixes, and today's taxonomy files every one of them. The router is Day 111's tool-choice pattern pointed at data sources.

Forward →: Tomorrow (Day 119) the capstone kicks off and your flowchart becomes a living runbook. Agentic retrieval gets its full treatment as the agent loop on Days 120–122, and Day 136 turns each taxonomy row into a metric.

Unlocks: D121 Agents II — Planning & Decomposition