Day 122 · The intern's notebook

Agents III — Memory & Context

You will be able to

Budget an agent's context window across system prompt, tools, history, and retrievals — in numbers
Implement history compaction: keep recent turns verbatim, roll older turns into a summary
Distinguish scratchpad, episodic, and semantic memory and pick storage for each
Wire retrieval-backed memory: store episode outcomes in a vector index and recall them for new tasks

Today's ~120 minutes

Spaced-rep: due cards + write a done_when for a sample subtask (D121)	10 min
ELI5 + tech read; context-window visualizer	20 min
Guided: drown-then-compact + episodic memory	45 min
Practice: the triage-agent context budget	18 min
Project: agent/memory.py with tests	20 min
Quiz + flashcards	7 min

Builds on: Day 120 — Agents I — the loop · Day 121 — Agents II — planning · Day 115 — Embeddings & vector databases · Day 104 — Context windows & lost-in-the-middle

The analogy

🧸 ELI5⚙ Tech

The intern's brain (the context window) holds maybe one desk's worth of paper at a time. On a long mission the papers pile up — every file she opened, every dead end — until the important sheet slides off the desk. The fix is not a bigger desk; even the biggest desk fills, and the middle of a tall pile is exactly where things get lost. The fix is a notebook system.

Three notebooks, actually. The scratchpad: today's working page — the current plan, the running total, the thing she's in the middle of. Torn out when the mission ends. The diary (episodic memory): one paragraph per finished mission — "checked settings.yaml for rate limits, found it; notes.txt is always stale." Tomorrow's missions start by skimming relevant diary entries. And the reference binder (semantic memory): distilled durable facts — "the config lives in settings.yaml" — no story attached, just what's true. When the desk fills mid-mission, she doesn't dump papers randomly: she summarizes the oldest ones onto one sheet ("mornings work: ruled out 3 files") and keeps the recent ones verbatim. The desk stays small; the mission stays whole.

Why this matters on the job

Context management is where real agents live or die: the model's per-call memory is the window and nothing else, every token costs money on every lap (history is re-sent each iteration — cost grows quadratically with naive accumulation), and Day 104 taught you the middle of a stuffed window is a dead zone. Long-running agents that "get dumber over time" are almost always drowning in their own history. This is also this week's RAG payoff: retrieval-backed memory is literally your Day 115 vector store pointed at the agent's own past. Claude Code, cursor-style agents, and every support bot with "memory" run exactly the machinery you build today.

Watch it happen

The intern's notebook — a fixed context window under pressure

step 1 / 6

memo

SYS

u1

a1

·

·

·

·

·

The context window is working memory: everything the model can "see" this call — here, a toy 8-slot window. The system prompt occupies the front permanently.

🎯 Predict modespeed1×

Guided practice

guided 1

Watch it drown, then build the compactor

30 min

Create agent_memory.py with the starter code. Part 1 simulates a 12-step mission with a naive append-everything history and a 900-token budget (tiny on purpose). The printed table shows tokens-per-lap climbing until the budget breaks mid-mission — and the cumulative-token column shows the O(N²) bill shape.
Part 2 adds ContextManager: last 3 turns verbatim, older turns rolled into a summary (scripted extractive summarizer standing in for the LLM call — the compaction PROMPT it replaces is printed). Rerun: tokens-per-lap now plateau under budget for the same 12 steps.
Verify the safety property: the file name "settings.yaml" and error code "ERR-4092" appearing in early turns must still appear VERBATIM in the compacted context at lap 12 (the code asserts this). Break it: make the summarizer drop identifiers and watch the assert fail — that corrupted-summary bug is real and nasty.
Compute from the printout: total tokens sent across the mission, naive vs compacted. Write the ratio in your notes — that is money.

🐍 python — editable, runs in your browser

# agent_memory.py — context budgeting + compaction.
def toks(s): return max(1, len(s) // 4)          # chars/4 ~ tokens (D96)

SYSTEM = "You are the Nimbus ops agent. Investigate using tools; cite files."
STEPS = [  # a 12-step mission's (action, observation) pairs, pre-scripted
    (f"step_{i}", obs) for i, obs in enumerate([
        "list_files -> [app.py, settings.yaml, notes.txt, legacy/, deploy.md]",
        "read_file(app.py) -> imports settings; serves /v1/query",
        "read_file(settings.yaml) -> rate_limit: 120  # ERR-4092 when exceeded",
        "grep('quota') -> deploy.md: 'quota resets monthly'",
        "read_file(notes.txt) -> TODO move rate limiting to gateway",
        "read_file(deploy.md) -> deploy via compose; quota resets monthly",
        "list_files(legacy/) -> [old_limits.cfg]",
        "read_file(legacy/old_limits.cfg) -> rate_limit: 60 (SUPERSEDED)",
        "grep('4092') -> settings.yaml comment; docs/api.md missing",
        "check_docs -> api docs not in repo; only settings comment",
        "compare -> current limit 120, legacy 60, superseded",
        "draft_answer -> settings.yaml sets rate_limit 120; ERR-4092 = exceeded",
    ], start=1)
]
BUDGET = 180

print("--- naive: append everything ---")
hist, total = [], 0
for i, (a, obs) in enumerate(STEPS, 1):
    hist.append(f"{a}: {obs}")
    ctx = SYSTEM + " ".join(hist)
    total += toks(ctx)
    flag = "  << OVER BUDGET" if toks(ctx) > BUDGET else ""
    print(f"lap {i:2d}: context={toks(ctx):4d} tok  cumulative={total:5d}{flag}")

class ContextManager:
    def __init__(self, keep_verbatim=3):
        self.recent, self.summary, self.keep = [], "", keep_verbatim
    def add(self, turn):
        self.recent.append(turn)
        while len(self.recent) > self.keep:
            self.summary = self._summarize(self.summary, self.recent.pop(0))
    def _summarize(self, summary, turn):
        # scripted stand-in for: "Fold this step into the running summary.
        # Keep decisions, facts, dead ends. PRESERVE identifiers exactly."
        keep = [w for w in turn.replace(",", " ").split()
                if "." in w or "_" in w or w.isupper() or w.isdigit()]
        return (summary + " | " + " ".join(keep[:6]))[-600:]
    def context(self):
        return SYSTEM + " SUMMARY:" + self.summary + " RECENT: " + " ".join(self.recent)

print("\n--- compacted: recent-verbatim + rolling summary ---")
cm, total2 = ContextManager(), 0
for i, (a, obs) in enumerate(STEPS, 1):
    cm.add(f"{a}: {obs}")
    total2 += toks(cm.context())
    print(f"lap {i:2d}: context={toks(cm.context()):4d} tok  cumulative={total2:5d}")

final = cm.context()
assert "settings.yaml" in final and "ERR-4092" in final, "identifiers lost!"
print(f"\nidentifiers preserved verbatim: OK")
print(f"total tokens  naive={total}  compacted={total2}  saved={100*(1-total2/total):.0f}%")

Ctrl/⌘+Enter runs · Tab indents · numpy/pandas/sklearn auto-load on import (torch and network calls need a local run)

guided 2

Episodic memory on the Day 115 vector store

15 min

Append the starter code below (it reuses your VecStore contract from Day 115 with the bag-of-words embedder — swap Chroma in locally).
Run it: three finished missions are written to memory as episode summaries with metadata. Then a NEW task arrives — "investigate why API calls fail after the 20th of the month" — and the agent retrieves relevant past episodes before starting.
Confirm the retrieved episode is the quota one (vocabulary overlap on quota/monthly/API), and note what gets injected into the system prompt: the LESSON, not the whole transcript. Memory injection competes with everything else for the context budget you just engineered.
Promote a fact: the quota lesson has now appeared twice (episodes 1 and 3). Write it into a SEMANTIC dict as a durable fact with a source-episode list — that promotion step (in production, a periodic LLM distillation pass) is how diaries become reference binders.
Forget something: delete the episode about the superseded legacy config (its lesson is now wrong). Note this is Day 115's deletability argument, aimed at the agent's own past.

🐍 python — editable, runs in your browser

episodes = [
    ("ep-1", "Task: debug ERR-4092 spike. Outcome: quota resets monthly; heavy "
             "clients hit limits near month end. Lesson: check quota calendar first.",
     {"task_type": "api-debug", "outcome": "solved"}),
    ("ep-2", "Task: find vacation rollover rule. Outcome: handbook hr#vacation "
             "has it. Lesson: HR questions -> handbook source filter.",
     {"task_type": "policy-qa", "outcome": "solved"}),
    ("ep-3", "Task: monthly API failure report. Outcome: failures cluster after "
             "the 20th; correlated with quota exhaustion. Lesson: quota again.",
     {"task_type": "api-debug", "outcome": "solved"}),
]
# store: VecStore from D115 (bag-of-words embed stands in for real embeddings)
import math, re
from collections import Counter
def embed(t): return Counter(re.findall(r"[a-z0-9-]+", t.lower()))
def cos(a, b):
    dot = sum(a[w] * b[w] for w in a if w in b)
    na, nb = math.sqrt(sum(v*v for v in a.values())), math.sqrt(sum(v*v for v in b.values()))
    return dot / (na * nb) if na and nb else 0.0

MEMORY = {eid: (embed(text), text, meta) for eid, text, meta in episodes}

def recall(task, k=1):
    qv = embed(task)
    ranked = sorted(((cos(qv, v), eid, text) for eid, (v, text, _) in MEMORY.items()),
                    reverse=True)
    return ranked[:k]

new_task = "investigate why API calls fail after the 20th of the month"
for score, eid, text in recall(new_task, k=2):
    print(f"{score:.3f} {eid}: {text[:70]}")

best = recall(new_task, k=1)[0]
lesson = best[2].split("Lesson:")[-1].strip()
print("\ninjected into system prompt -> 'Relevant experience:", lesson + "'")

SEMANTIC = {"quota-calendar": {"fact": "API quota resets monthly; failures cluster late-month",
                               "sources": ["ep-1", "ep-3"]}}
del MEMORY["ep-2"]  # pretend it went stale: forgetting is an operation, not an accident
print("semantic facts:", list(SEMANTIC), "| episodes remaining:", list(MEMORY))

Ctrl/⌘+Enter runs · Tab indents · numpy/pandas/sklearn auto-load on import (torch and network calls need a local run)

On your own

Write the budget, then defend it

18 min

Your Day 126 triage agent will run on a 32k-token window with: a 1200-token system prompt + tool definitions, tickets up to 800 tokens, RAG context (Day 119 retriever) of 3 chunks × 400 tokens, up to 2 recalled episodes × 150 tokens, and a required 1000-token response reserve. Produce context_budget.md: (1) the line-item budget with a history allowance and an explicit safety margin; (2) the compaction trigger (at what history size do you compact, and to what); (3) which line you would cut first under pressure and why; (4) the two things that must NEVER be compacted.

Hints: budgets that sum to exactly the window are already broken — retrievals vary. Plan compaction to trigger well before the ceiling, not at it.

Ship before you stop

Memory module for the Week 18 agent

Extend the agent package: agent/memory.py with (1) ContextManager from the lab, parameterized by token budget and keep_verbatim, with the identifier-preservation property under test; (2) EpisodeStore wrapping your Day 115 VecStore: record(task, outcome, lesson, meta) after missions, recall(task, k) before them, forget(episode_id); (3) integration: run_agent (Day 120) gains an optional memory argument — recalled lessons injected into the system section, episode recorded at mission end. Tests: compaction keeps context under budget across a scripted 15-turn mission; identifiers survive compaction; recall returns the planted relevant episode; forget removes it. Update the README mapping (compaction call, distillation pass) to their real-LLM equivalents.

Rubric — check what you completed (0/6)

ContextManager enforces the token budget on every lap of a 15-turn test missionIdentifier-preservation assert passes (file names, error codes verbatim post-compaction)EpisodeStore record/recall/forget all covered by tests against planted episodesrun_agent integration injects lessons and records episodes without changing loop logiccontext_budget.md committed alongside with the four required sectionsCommitted to the practice repo

Common mistakes & misconceptions

Solving context pressure with a bigger window. Cost scales with tokens re-sent per lap, and lost-in-the-middle worsens with stuffing — budget and compact instead.
Summarizing away exact identifiers. A paraphrased file name or error code is corrupted state; summaries must preserve IDs verbatim — assert it in tests.
One memory bucket for everything. Scratchpad (this mission), episodic (past missions), semantic (durable facts) have different lifetimes, stores, and injection points.
Injecting whole transcripts as memory. Recall lessons, not logs — memory competes with retrievals and history for the same budget you wrote this morning.
Never forgetting. Stale lessons misdirect future missions; contradiction should tombstone (Day 115's deletability aimed at the agent's own past). Forgetting is an operation.
Compacting the plan state or system prompt. The working plan (Day 121) and the agent's instructions are load-bearing every lap; compaction applies to history only.

Knowledge check

Q1. Why does a naive append-everything agent loop cost O(N²) tokens over an N-step mission?

Q2. Your compactor summarizes "read legacy/old_limits.cfg, found rate_limit 60, superseded" as "checked an old config." What breaks later?

Q3. Where does "the intern learns from experience" live, mechanically?

Go deeper — curated resources

articleLilian Weng — LLM Powered Autonomous Agents (memory section) ↗25 min articleAnthropic — Building Effective Agents (augmented LLM: retrieval & memory) ↗15 min docsLangChain docs — short-term & long-term memory ↗15 min

If you have a third hour

MemGPT / letta-style memory hierarchies — Treats the context window like RAM and external stores like disk, with the agent paging its own memory via tools — today's architecture taken to its logical extreme.

Done means

Naive-vs-compacted token totals measured; savings ratio recorded
Identifier-preservation assert understood by breaking it once
Episodic recall demo run: right episode recalled, lesson (not transcript) injected
context_budget.md written; agent/memory.py committed with tests green
Quiz ≥ 2/3

How this connects

← Back: Day 104 named the disease (finite windows, dead middles); Day 115's vector store turned out to be the memory organ; Day 121's plan table was a scratchpad before you had the word for it.

Forward →: Day 125 adds guards around this now-complete agent, and Day 126's triage agent runs on today's budget doc. Day 142's traces will show you context sizes per lap in production.

Unlocks: D125 Agent Reliability · D126 Week 18 Checkpoint: Support-Triage Agent