Day 115 · The library with a meaning-based index

Embeddings & Vector Databases

You will be able to

Choose an embedding model deliberately (dimensions, domain, cost, and the lock-in it creates)
Explain exact vs approximate nearest-neighbor search and the HNSW recall/speed trade
Build and query a vector index with Chroma-style code: add, upsert, metadata filters
Pick the right distance metric and say when cosine vs dot product vs Euclidean differ

Today's ~125 minutes

Spaced-rep: due cards + explain chunk-size trade-off aloud (D114)	10 min
ELI5 + tech read; vector-search visualizer (watch HNSW hop)	25 min
Guided: VecStore contract + Chroma lab	40 min
Practice: find the exact-search cliff	20 min
Project: build_index.py dry run	20 min
Quiz + flashcards	10 min

Builds on: Day 113 — RAG architecture · Day 92 — Embeddings — meaning as geometry · Day 50 — Vectors & cosine similarity

The analogy

🧸 ELI5⚙ Tech

A normal library index answers "where is the book titled X?" — you must know the exact title. A meaning-based library is stranger and better: every book's CONTENT is boiled down to a point on a giant map, where books about similar things sit near each other. To find "something about taking time off work," you boil your question down to a point on the same map and grab the nearest neighbors — even if no book contains your exact words. "Vacation policy" sits right next to "time off," because the map was drawn from meaning, not spelling.

The map is the embedding space (Day 92's map of meaning). The vector database is the librarian who can find nearest neighbors FAST: with a million books, checking the distance to every single one per question is too slow, so the librarian builds shortcut walkways between neighborhoods (an ANN index like HNSW) and hops along them — visiting maybe a thousand books instead of a million, and almost always finding the true nearest ones.

Why this matters on the job

This is the O(n²)/O(n) moment from Day 22 arriving in production: exact search over every chunk is linear per query and fine at 10k chunks, but a 20M-chunk enterprise corpus at 50 queries/sec needs ANN indexes — knowing WHERE that line sits is an interview staple and a real architecture decision you'll defend to customers. Just as important is the operational side FDEs live in: embedding-model version lock-in (change the model, re-embed everything), metadata filtering for per-customer access control, and upsert lifecycles for docs that change daily.

Watch it happen

The meaning-based library — ANN candidate buckets, then a careful rerank

step 1 / 6

refund policybilling faq

[0]

leave policyholidaysbenefits

[1]

api docssdk guide

[2]

sso setupsecurity

[3]

A vector index at rest: document chunks clustered into cells by MEANING — billing chunks together, HR together. Comparing a query against all million chunks would work, but at answer-the-user speed it's too slow.cells = neighborhoods in embedding space (IVF / HNSW regions)

🎯 Predict modespeed1×

Guided practice

guided 1

A vector store from scratch — the contract

20 min

Create vecstore.py with the starter code: a tiny class with the EXACT api shape of a real vector DB — add, upsert, query with where filters — over brute-force cosine. Runs in the browser interpreter.
Run it. Note the three behaviors the tests demonstrate: (a) top-k ranking; (b) a where filter restricting results to handbook docs; (c) an upsert replacing an old vector under the same ID — the doc-changed lifecycle.
Predict, then verify: after upserting hb-01 with new text, does the old text ever come back? Why is a STABLE id scheme what made replacement (not duplication) happen?
This class IS the mental model: everything a vector DB adds beyond it is speed (HNSW), persistence, and scale.

🐍 python — editable, runs in your browser

import math

class VecStore:
    """Minimal vector store: the contract of Chroma/pgvector/Qdrant,
    minus the speed. Brute-force cosine = exact search, fine to ~100k."""
    def __init__(self):
        self.rows = {}   # id -> (vector, text, metadata)

def add(self, id, vector, text, metadata):
        if id in self.rows:
            raise ValueError(f"duplicate id {id} — use upsert")
        self.rows[id] = (vector, text, metadata)

def upsert(self, id, vector, text, metadata):
        self.rows[id] = (vector, text, metadata)   # replace or insert

@staticmethod
    def _cosine(a, b):
        dot = sum(x * y for x, y in zip(a, b))
        na = math.sqrt(sum(x * x for x in a))
        nb = math.sqrt(sum(x * x for x in b))
        return dot / (na * nb) if na and nb else 0.0

def query(self, qvec, k=3, where=None):
        hits = []
        for id, (vec, text, meta) in self.rows.items():
            if where and any(meta.get(key) != val for key, val in where.items()):
                continue                      # metadata filter
            hits.append((self._cosine(qvec, vec), id, text, meta))
        return sorted(hits, reverse=True)[:k]

# --- toy embeddings: 4 dims = (hr-ness, security-ness, product-ness, money-ness)
# Real models produce 384-3072 dims; the geometry lesson is identical.
E = {
    "vacation days roll over up to 10":        [0.9, 0.0, 0.0, 0.2],
    "parental leave is 16 weeks paid":         [0.95, 0.0, 0.0, 0.1],
    "laptops must use full-disk encryption":   [0.1, 0.9, 0.0, 0.0],
    "API rate limit is 120 requests/minute":   [0.0, 0.1, 0.9, 0.0],
    "purchases over 500 USD need approval":    [0.3, 0.0, 0.1, 0.9],
}
store = VecStore()
for i, (text, vec) in enumerate(E.items()):
    src = "handbook" if vec[0] > 0.2 or vec[3] > 0.5 else "techdocs"
    store.add(f"chunk-{i}", vec, text, {"source": src})

q_timeoff = [0.85, 0.0, 0.0, 0.15]   # "how much time off do I get?"
print("no filter:      ", [(round(s,3), t[:30]) for s, _, t, _ in store.query(q_timeoff)])
print("handbook only:  ", [(round(s,3), t[:30]) for s, _, t, _ in store.query(q_timeoff, where={"source": "handbook"})])

# --- the upsert lifecycle: hb policy changed ---
store.upsert("chunk-0", [0.9, 0.0, 0.0, 0.2],
             "vacation days roll over up to 15 (updated 2026)", {"source": "handbook"})
print("after upsert:   ", [(round(s,3), t) for s, _, t, _ in store.query(q_timeoff, k=1)])
print("rows in store:  ", len(store.rows), "(still 5 — replaced, not duplicated)")

Ctrl/⌘+Enter runs · Tab indents · numpy/pandas/sklearn auto-load on import (torch and network calls need a local run)

guided 2

The real thing: Chroma over your Day 114 chunks

20 min

This exercise needs a local Python environment (pip install chromadb sentence-transformers) — the in-browser interpreter can't run it; if you're browser-only today, read the code line by line and map each call onto your VecStore, then run it tonight.

Create chroma_lab.py with the starter code. It ingests the structure-aware chunks from Day 114's policy doc into Chroma, which embeds them automatically (all-MiniLM-L6-v2, 384 dims, runs on CPU).
Run it. The killer test is the last query: "how much time off do I get" — the vocabulary-mismatch query your Day 113 bag-of-words retriever FAILED. Real embeddings place "time off" near "vacation." Confirm it now retrieves the right chunk.
Try the where filter — then break it: filter for a source that doesn't exist and see the graceful empty result.
Re-run the script. Note get_or_create_collection + stable IDs make re-ingest idempotent — accidental double-ingest is the classic index-poisoning bug.

🐍 python — editable, runs in your browser

# Local run: pip install chromadb  (embeds with a local model by default)
import chromadb

client = chromadb.PersistentClient(path="./chroma_data")
col = client.get_or_create_collection(
    name="nimbus_docs",
    metadata={"hnsw:space": "cosine"},     # metric fixed at creation!
)

chunks = [
    ("expenses#purchases#0", "Expenses > Purchases: purchases over 500 USD require manager pre-approval. Software goes through IT procurement.", {"source": "handbook", "section": "Purchases"}),
    ("expenses#travel#0",    "Expenses > Travel: book through the Navan portal. Hotels capped at 250 USD/night tier-1, 180 elsewhere.",          {"source": "handbook", "section": "Travel"}),
    ("hr#vacation#0",        "HR > Vacation: employees accrue 1.5 vacation days per month; up to 10 unused days roll over.",                     {"source": "handbook", "section": "Vacation"}),
    ("api#limits#0",         "API > Limits: the Insights API allows 120 requests per minute. ERR-4092 means monthly quota exhausted.",           {"source": "techdocs", "section": "Limits"}),
]
col.upsert(   # stable ids => re-running this script never duplicates
    ids=[c[0] for c in chunks],
    documents=[c[1] for c in chunks],
    metadatas=[c[2] for c in chunks],
)

for q in ["hotel budget for a work trip",
          "what does ERR-4092 mean",
          "how much time off do I get"]:      # D113's failing query
    res = col.query(query_texts=[q], n_results=2)
    print("Q:", q)
    for doc, dist in zip(res["documents"][0], res["distances"][0]):
        print(f"   dist={dist:.3f}  {doc[:70]}")

res = col.query(query_texts=["spending rules"], n_results=3,
                where={"source": "handbook"})   # metadata filter
print("handbook-only:", [d[:40] for d in res["documents"][0]])

Ctrl/⌘+Enter runs · Tab indents · numpy/pandas/sklearn auto-load on import (torch and network calls need a local run)

On your own

Find the exact-search cliff

20 min

Using your pure-Python VecStore, measure where brute force stops being fine. Generate synthetic corpora of n = 1_000, 10_000, 100_000 random 384-dim vectors (random.random()), time 20 queries against each (reuse Day 22's timing harness), and extrapolate: at what n does a single query exceed 100 ms? Then write a 5-line recommendation: for the capstone corpus (~2k chunks), is an ANN index justified on latency grounds — or is the honest answer "exact search, revisit at 100k"?

Hints: per-query cost is O(n·d) — confirm the linear ratio between the three sizes. Python loops are ~100x slower than NumPy/real DBs; note that your cliff estimate is conservative and say by how much.

Ship before you stop

Index the capstone corpus (dry run)

Build build_index.py: an ingest script that takes a folder of markdown files (create 4–6 fake Nimbus docs, or reuse Day 114's), applies your structure-aware chunker with the Day 114 decision-card defaults, stamps metadata (source, section path, position, model name "bow-v0"), and loads everything into your VecStore — plus a Chroma variant behind an if USE_CHROMA flag for local runs. Include a smoke test: 5 queries with expected source sections, asserting the right section appears in top-2. This script IS the seed of your capstone's ingest pipeline; on Day 119 you will point it at the real corpus.

Rubric — check what you completed (0/6)

Ingest walks a folder, chunks with Day 114 defaults, and stamps all four metadata fieldsDeterministic chunk IDs (source#section#position) — re-running never duplicatesSmoke test: ≥ 4/5 queries retrieve the expected section in top-2Embedding model name recorded in metadata (the lock-in you now understand)Chroma variant present and runnable locallyCommitted to the practice repo

Common mistakes & misconceptions

Reaching for a hosted vector DB at 3k chunks. Exact search is O(n·d), perfect-recall, and instant at that scale — complexity must be earned by n, latency, or ops needs.
Mixing embedding model versions in one index. Vectors from different models/versions are geometrically incomparable; the symptom is quietly mediocre retrieval, not an error.
Forgetting ANN recall < 100%. HNSW can miss the true nearest chunk; if you never measure retrieval recall (Day 136), these misses masquerade as "the model is dumb."
Choosing dot product for unnormalized vectors without meaning to. Magnitude then influences ranking — long chunks win regardless of relevance. For unit vectors, cosine/dot/Euclidean rank identically.
Random or content-hash chunk IDs. Doc updates then ADD vectors instead of replacing them, and stale chunks keep answering. Deterministic IDs + upsert = the update lifecycle.
Applying metadata filters as a post-filter on ANN results. You can get back fewer than k (or zero) hits; engines must filter during traversal — know which yours does.

Knowledge check

Q1. You re-embed new docs with your embedding provider's v3 model into an index full of v2 vectors. What happens?

Q2. When is brute-force exact search the RIGHT engineering choice?

Q3. Your embedding model outputs unit-normalized vectors. Choosing cosine vs dot product vs Euclidean changes…

Go deeper — curated resources

docsChroma — getting started (client, collections, query) ↗20 min articlePinecone Learning Center — HNSW & vector indexes ↗25 min docsSentence-Transformers — pretrained embedding models ↗15 min docsQdrant docs — filtering & payloads ↗15 min

If you have a third hour

pgvector — vectors inside Postgres — If the customer already runs Postgres, one extension gives vectors + SQL + transactions with no new infra: a very common FDE recommendation. Compare its IVFFlat vs HNSW index options.

Done means

VecStore demonstrates add/upsert/query-with-filter; upsert lifecycle explained
Chroma lab run locally (or fully traced if browser-only) — the D113 failing query now retrieves correctly
Exact-search cliff measured with the linear ratio confirmed
build_index.py committed with passing smoke test
Quiz ≥ 2/3

How this connects

← Back: Day 50's cosine similarity and Day 92's map of meaning are now infrastructure. Day 22's growth-shape reasoning told you exactly when brute force dies — today you measured the cliff yourself.

Forward →: Day 116 fuses this dense index with BM25 because embeddings still miss exact identifiers. Day 119 points build_index.py at the capstone corpus, and Day 149 puts the vector DB into your docker-compose stack.

Unlocks: D116 Hybrid Search · D117 Reranking & Query Transforms · D119 Week 17 Checkpoint: Capstone Kickoff — Docs-QA v0 · D122 Agents III — Memory & Context