Day 128 · Sticky notes on the giant brain

Fine-Tuning II — LoRA Lab

You will be able to

Explain LoRA as freezing W and learning a low-rank update BA, and compute the parameter savings
Choose rank, alpha, and target modules with a defensible rationale
Run a small LoRA fine-tune with PEFT on an open model and watch it acquire a style
Compare the tuned model against base + few-shot prompting honestly, on held-out inputs
Explain QLoRA and adapter merging in one sentence each

Today's ~120 minutes

Spaced-rep warm-up + recall Day 52 low-rank intuition	10 min
ELI5 + tech read; LoRA parameter math on paper	20 min
Guided: NumPy LoRA math + real PEFT fine-tune	45 min
Practice: rank ablation	20 min
Project: the honest adapter report	15 min
Quiz + flashcards	10 min

Builds on: Day 52 — Low-rank approximation & SVD intuition · Day 88 — PyTorch training loops · Day 127 — When to fine-tune & SFT data

The analogy

🧸 ELI5⚙ Tech

Editing the polymath's entire brain is surgery: billions of connections, months of drilling, and you risk breaking what already works. LoRA is sticky notes instead. Leave every original connection untouched and frozen, and attach small removable notes at key junctions: "when the report question comes up, lean formal; open with the summary." The notes are tiny — a few million words of correction on a brain of billions — but placed at the right junctions they steer behavior convincingly.

The trick from Day 52 is why tiny works: the correction you need is usually low-rank. You are not teaching a new language; you are applying one consistent stylistic push, and a consistent push can be written as two thin matrices instead of one enormous one. Bonus: sticky notes peel off. Keep the base brain, swap note-sets per customer, and store each set in megabytes instead of re-copying the whole brain.

Why this matters on the job

LoRA is the reason fine-tuning is accessible at all: full fine-tuning a 7B model wants multiple 80GB GPUs; a LoRA (or QLoRA) run fits on one modest card or a free Colab. That changes the customer conversation — "we can try an adapter on your data this week" is a sentence an FDE can actually say. It also changes deployment: one base model serving many customers, each with their own few-MB adapter, is a real multi-tenant architecture. And interviewers love the question "why does low-rank adaptation work?" — Day 52 gave you the honest answer.

Watch it happen

Sticky notes on the giant brain — LoRA's low-rank arithmetic

step 1 / 5

	full fine-tune
W (4096×4096)	16,777,216 params updated
memory	weights + grads + optimizer ×2

One weight matrix in a big model: 4096 × 4096 = 16.7 million parameters. Full fine-tuning updates ALL of them — for every matrix in the model. That is GPU-farm territory.

🎯 Predict modespeed1×

Guided practice

guided 1

LoRA arithmetic in NumPy — see why tiny works

15 min

Create lora_math.py with the starter code and run it.
It builds a "pretrained" weight matrix W (512×512), a rank-8 update BA, and compares parameter counts: full update vs LoRA factors.
Then the Day 52 payoff: it takes a random full-rank update, computes its best rank-8 approximation via SVD, and prints the reconstruction error — versus the error for a update that is genuinely low-rank (error ~0).
Write two sentences in comments: why a "consistent stylistic push" is plausibly low-rank, and what kind of adaptation would NOT be (hint: learning a new language's worth of arbitrary facts).
Change r to 2, 8, 32 and record how approximation error and parameter count move.

🐍 python — editable, runs in your browser

import numpy as np

d = k = 512
r = 8
rng = np.random.default_rng(0)

W = rng.normal(size=(d, k))                     # frozen pretrained weights
B = rng.normal(size=(d, r)) * 0.01
A = rng.normal(size=(r, k)) * 0.01
full_params, lora_params = d * k, r * (d + k)
print(f"full update params: {full_params:,}")
print(f"LoRA params (r={r}): {lora_params:,}  "
      f"({100 * lora_params / full_params:.1f}% of full)")

# Day 52 payoff: best rank-r approximation of an update via SVD
def rank_r_error(delta, r):
    U, S, Vt = np.linalg.svd(delta, full_matrices=False)
    approx = (U[:, :r] * S[:r]) @ Vt[:r]
    return np.linalg.norm(delta - approx) / np.linalg.norm(delta)

random_update = rng.normal(size=(d, k))          # arbitrary, full-rank
lowrank_update = rng.normal(size=(d, r)) @ rng.normal(size=(r, k))
print(f"rank-{r} approx error, random full-rank update: "
      f"{rank_r_error(random_update, r):.3f}")   # ~0.99 — hopeless
print(f"rank-{r} approx error, truly low-rank update:  "
      f"{rank_r_error(lowrank_update, r):.3f}")  # ~0.0

# LoRA hypothesis: the *useful* fine-tuning update is closer to the
# second case than the first — one consistent push, not d*k arbitrary edits.

Ctrl/⌘+Enter runs · Tab indents · numpy/pandas/sklearn auto-load on import (torch and network calls need a local run)

guided 2

Run a real LoRA fine-tune with PEFT

30 min

Install: pip install torch transformers peft datasets (CPU is fine for this model size; expect a few minutes of training).
The starter trains distilgpt2 (82M params) to adopt an unmistakable style: answers that begin with "VERDICT:" and end with "— so ruled." Tiny model, tiny corpus — the point is watching an adapter take hold, not quality.
Run it. Note the printed trainable-parameter percentage (should be well under 1%).
Generate from the BASE model and the TUNED model on the same three held-out prompts. The base rambles; the tuned one adopts the format.
Now the honest baseline: prompt the BASE model few-shot with two style examples in-context. Compare all three outputs. On a task this simple, few-shot may tie the tune — write one sentence on when that stops being true (prompt length, consistency at scale).
Save the adapter (model.save_pretrained) and check its size on disk versus the base model.

🐍 python — editable, runs in your browser

import torch
from datasets import Dataset
from transformers import (AutoModelForCausalLM, AutoTokenizer,
                          Trainer, TrainingArguments,
                          DataCollatorForLanguageModeling)
from peft import LoraConfig, get_peft_model

model_name = "distilgpt2"
tok = AutoTokenizer.from_pretrained(model_name)
tok.pad_token = tok.eos_token
base = AutoModelForCausalLM.from_pretrained(model_name)

cfg = LoraConfig(r=8, lora_alpha=16, target_modules=["c_attn"],
                 lora_dropout=0.05, task_type="CAUSAL_LM")
model = get_peft_model(AutoModelForCausalLM.from_pretrained(model_name), cfg)
model.print_trainable_parameters()   # ~0.2-0.5% of total

style = [("Is tea better than coffee?",
          "VERDICT: Tea, for the patient. Coffee, for the deadline. — so ruled."),
         ("Should I refactor this function?",
          "VERDICT: If you feared touching it, yes. — so ruled."),
         ("Is Python slow?",
          "VERDICT: Slower than C, faster than the meeting about C. — so ruled."),
         ("Do I need a vector database?",
          "VERDICT: Not before you need vectors. — so ruled."),
         ("Should errors be silent?",
          "VERDICT: Never. Loud errors are cheap; silent ones compound. — so ruled.")] * 8

def fmt(qa):
    return {"text": f"Q: {qa[0]}\nA: {qa[1]}{tok.eos_token}"}
ds = Dataset.from_list([fmt(x) for x in style]).map(
    lambda b: tok(b["text"], truncation=True, max_length=64), batched=True)

trainer = Trainer(
    model=model,
    args=TrainingArguments("lora_out", num_train_epochs=8,
                           per_device_train_batch_size=4,
                           learning_rate=2e-4, logging_steps=10,
                           report_to=[]),
    train_dataset=ds,
    data_collator=DataCollatorForLanguageModeling(tok, mlm=False))
trainer.train()

def gen(m, prompt):
    ids = tok(f"Q: {prompt}\nA:", return_tensors="pt")
    out = m.generate(**ids, max_new_tokens=40, do_sample=False,
                     pad_token_id=tok.eos_token_id)
    return tok.decode(out[0], skip_special_tokens=True)

for p in ["Is documentation worth writing?", "Should I use recursion here?"]:
    print("BASE :", gen(base, p))
    print("TUNED:", gen(model, p), "\n")
model.save_pretrained("adapter_verdict")   # check the size: a few MB

Ctrl/⌘+Enter runs · Tab indents · numpy/pandas/sklearn auto-load on import (torch and network calls need a local run)

On your own

Rank ablation, honestly reported

20 min

Re-run the guided fine-tune at r = 2, 8, and 32 (same epochs, same seed). For each: record trainable-parameter count, final training loss, and — the part that matters — format adherence on FIVE held-out prompts you write yourself (does the output start with VERDICT: and end with — so ruled?). Score adherence out of 5.

Deliver a small table (r, params, loss, adherence/5) and a three-sentence conclusion: where did capacity saturate, did r=32 buy anything, and what would you try next if adherence plateaued below 5/5 (more epochs? more diverse examples? more target modules?).

Hints: training loss falling is NOT the metric — held-out adherence is. If all ranks tie, say so; "no effect at this scale" is an honest, publishable result.

Ship before you stop

The honest adapter report

Write lora_report.md in your practice repo documenting today's lab like an engineer, not a fan: (1) setup — model, adapter config, dataset size; (2) the three-way comparison on five held-out prompts — base zero-shot, base few-shot, LoRA-tuned — as a table with format-adherence scores; (3) parameter and disk-size math — trainable %, adapter MB vs base MB; (4) the rank-ablation table from practice; (5) a verdict paragraph: for THIS task, was the adapter worth it over few-shot, and what task profile (long prompts, strict format at high volume, per-tenant styles) would flip the answer. Commit alongside the code and adapter.

Rubric — check what you completed (0/6)

Three-way comparison includes base + FEW-SHOT, not just base zero-shotAll comparisons are on held-out prompts, none from training dataParameter math is correct and stated (trainable %, r×(d+k) reasoning)Rank ablation table present with a stated conclusionVerdict names the task profile where the adapter beats prompting, and where it does notCommitted with code and saved adapter

Common mistakes & misconceptions

Evaluating on training prompts. A 40-example adapter memorizes; only held-out inputs measure learning. Same leakage law as Day 127.
Comparing the tune to a weak baseline. Beat base + best few-shot prompt, or the tune has not paid for itself.
Treating training-loss decrease as success. Loss can fall while held-out behavior is unchanged (or worse); score the behavior you wanted.
Cranking rank first. r=8 often saturates style/format tasks; if quality is short, more/better data and more epochs usually beat r=64.
Forgetting alpha scales the update by alpha/r — changing r with alpha fixed silently changes the effective learning-rate of the adapter. Change one knob at a time.
Assuming the adapter added knowledge. It restyled outputs; ask it a fact question and it hallucinates exactly like base. Knowledge is still RAG's job.

Knowledge check

Q1. For a 1024×1024 weight matrix, a rank-8 LoRA adapter trains how many parameters (ignoring alpha)?

Q2. Why can a low-rank update capture a fine-tune's effect at all?

Q3. Your LoRA model nails the format on the 40 training questions but reverts to rambling on new ones. Diagnosis?

Go deeper — curated resources

paperLoRA paper — Hu et al. 2021 ↗30 min docsHugging Face PEFT documentation ↗25 min courseHugging Face LLM Course ↗20 min

If you have a third hour

QLoRA paper (Dettmers et al. 2023) ↗ — NF4 quantization + double quantization + paged optimizers — the engineering that made 65B fine-tuning fit on one 48GB card.

Done means

lora_math.py run; SVD low-rank demonstration understood and annotated
PEFT fine-tune completed; adapter saved and size compared to base
Three-way comparison (base / few-shot / tuned) done on held-out prompts
Rank ablation table committed in lora_report.md
Quiz ≥ 2/3

How this connects

← Back: Day 52 promised low-rank approximation would pay off — today it did, literally as the SVD error you printed. The Trainer loop is Day 88's canonical loop wearing a library; the honest-baseline discipline is Day 127.

Forward →: Day 129 asks the next question: instead of adapting a big model, when does a small or quantized one win outright? QLoRA's 4-bit trick gets its full explanation there. Your before/after comparison instinct becomes a formal harness on Days 134–140.

Unlocks: D129 Distillation, Quantization & Model Selection