LLM APIs I — The Request
- Make a first scripted request to a frontier model with the Anthropic Python SDK
- Explain the anatomy of a chat request: model, max_tokens, system, and the messages array
- Demonstrate that the API is stateless by building a multi-turn chat loop that resends history
- Read usage fields and stop_reason from responses and log them on every call
- Translate a request between the Anthropic SDK shape and the OpenAI-compatible shape
| Spaced-rep warm-up: Phase 5 repair cards from Day 105 | 10 min |
| ELI5 + tech read: request anatomy & statelessness | 20 min |
| Guided: first request + response autopsy | 20 min |
| Guided: statelessness proof + chat loop | 20 min |
| Practice: OpenAI-compatible dialect against Ollama | 20 min |
| Project: llm_chat.py + quiz + flashcards | 30 min |
Builds on: Day 100 — Training pipeline — what an assistant model is · Day 96 — Tokenization & counting · Day 41 — HTTP & APIs
For three weeks you studied the polymath's brain. Today you rent one. The deal is beautifully simple: you send a structured letter, you get a structured reply, you pay by the word — in both directions. The letter has three parts. The briefing memo (system prompt) sets the job: "You are a support assistant for Acme; be concise; never discuss competitors." The conversation transcript (messages) is the dialogue so far, labeled by speaker. And a budget cap (max_tokens) says "reply with at most this much."
Here is the part that surprises everyone: the polymath has no memory of you. None. Every letter arrives as if from a stranger, so if you want a conversation, YOU must include the entire transcript so far in every letter — your questions and the polymath's own previous answers. The "memory" in every chat product you have ever used is just the app diligently resending history. Forget this and your bot develops amnesia mid-conversation; remember it and you also understand why long conversations cost more per turn: the letter keeps growing, and you pay by the word.
This request shape is the atom of the entire AI-engineering stack: RAG (Day 113) is stuffing retrieved text into messages, agents (Day 120) are loops around this call, evals (Day 134) replay suites of these calls. Statelessness is the single most common conceptual bug in beginners' LLM apps — history mismanagement causes both amnesia bugs and runaway token bills. And usage-field literacy from day one is how you avoid the classic startup moment: discovering the month's API bill funded a feature nobody priced. Every later cost, latency, and context decision reads from what you log today.
Guided practice
First request + response autopsy
20 minpip install anthropicin a fresh venv. ExportANTHROPIC_API_KEY(create a key in the provider console; treat it like a password — Day 44).- Paste the starter and run it. Read every printed field.
- Autopsy: what is response.id? What model string came back? Why is content a list?
- Set max_tokens=15 and rerun. Observe the truncated text and stop_reason == 'max_tokens'. Write one line on why production code must check stop_reason.
- Add stop_sequences=['5.'] and ask for a 10-item list. Observe early stopping and the stop_reason.
- Compute the request cost from usage at illustrative list prices (3.00 per million input tokens, 15.00 per million output tokens): cost = in_tok * 3.00 / 1e6 + out_tok * 15.00 / 1e6. Print it with 6 decimal places.
Prove statelessness, then build memory
20 min- Send 'My name is Priya and I work on billing infra.' as one request. Then send 'What is my name?' as a SEPARATE fresh request. Confirm the model has no idea — statelessness demonstrated.
- Now paste the chat-loop starter: it appends each user turn AND each assistant reply to a history list and resends everything.
- Repeat the two questions inside the loop — now it remembers. You built the memory.
- Watch input_tokens grow each turn as history accumulates. After 5 turns, note the growth pattern and explain it in one line.
- Bonus: print a running total cost across the session.
On your own
Speak both dialects
20 minPort your chat loop to the OpenAI-compatible shape and run it against a local server: start ollama run llama3.2:1b (Day 103), then point the openai SDK at http://localhost:11434/v1 with any api_key string.
Goals: (1) working chat loop against the local endpoint; (2) a short dialect-diff table in your notes: where the system prompt lives, method name, response text path, usage field names; (3) one sentence on what stayed identical (statelessness, alternating roles, pay-by-token concept).
Hints: OpenAI-shape puts {'role': 'system'} as the first element of messages; the reply is at resp.choices[0].message.content; Ollama's compatible endpoint accepts model='llama3.2:1b'.
llm_chat.py — your first API artifact
Polish the chat loop into llm_chat.py in a new llm-toolkit repo you will grow all phase: argparse flags for --system and --max-tokens (Day 16), a /reset command that clears history, a /usage command printing session token totals and illustrative cost, graceful handling of a missing API key (clear error, non-zero exit), and every exchange appended as JSON lines to chat_log.jsonl (timestamp, model, tokens, stop_reason). README documents the stateless-history design decision. Commit and push — Day 107 upgrades this exact file with streaming and retries.
Common mistakes & misconceptions
- Assuming the API remembers the conversation. It is stateless; the client owns history. Every "my bot forgot the context" bug traces here.
- Forgetting to append the assistant's reply to history. The model then sees only user turns — a subtly broken transcript that degrades answers before it visibly breaks.
- Hardcoding the API key in source. Env vars locally, secret managers in production (Day 44/159). Keys in git history are compromised keys.
- Ignoring stop_reason. A max_tokens truncation silently ships half an answer to your user unless you check and handle it.
- Treating max_tokens as a suggestion. It is a hard cap covering the model's internal reasoning plus visible text on current models — undersizing it truncates good answers.
- Pricing features from vibes instead of usage fields. Log tokens from call one; Day 107 turns those logs into real cost engineering.
Q1. Why must you resend the full message history on every turn?
Q2. A response comes back with stop_reason "max_tokens". What happened and what should production code do?
Q3. In the OpenAI-compatible dialect, the Anthropic top-level system parameter corresponds to…
Go deeper — curated resources
- Count before you send — The API offers a token-counting endpoint (client.messages.count_tokens) — try measuring a prompt before sending it, and compare with the usage fields after. Useful for pre-flight budget checks in Day 122's context engineering.
- First request ran with all metadata fields read and explained
- Statelessness demonstrated, then solved with the history loop
- Both dialects exercised with the diff table written
- llm_chat.py pushed with logging, /usage, /reset, and truncation detection
- Quiz ≥ 2/3
← Back: You now rent what Days 99-100 taught you to build: the messages array is the chat template from Day 103 worn on the outside, and the token meter is Day 96's counting with a price attached.
Forward →: Day 107 hardens this exact code with streaming, retries, and cost engineering. Day 108's prompt engineering fills the system field you used today; Day 111 adds tools to this same call; and by Day 119 this request shape sits at the heart of your capstone.
Unlocks: D107 LLM APIs II — Streaming, Retries & Cost · D108 Prompt Engineering I — The Briefing Memo · D111 Tool Use & Function Calling · D113 RAG I — Architecture