How prompt caching reuses an identical prefix, why one timestamp breaks it, and how to keep long conversations and large inputs inside the context window.
In 30 seconds
Prompt caching reuses an identical prompt prefix: tools, then system prompt, then messages.
Stable content first, breakpoint after it — any change before the breakpoint is a miss.
Keep context focused: retrieve relevant sections, trim stale tool results, compact long tasks.
In big codebases, search first and read only what matches.
Caching matches the prompt from the start. Put stable content first and the breakpoint after it; one changing value early breaks everything behind it.
How caching decides what to reuse
Many requests start the same way: the same tool definitions, the same long system prompt, perhaps the same reference document. Prompt caching lets those requests reuse the processed prefix instead of processing it again, which lowers cost and latency. The prefix is compared from the very beginning, in order — tools, system prompt, then messages — so it must be byte-for-byte identical up to the cache breakpoint.
That is why a timestamp, a request ID or a user's name at the top of the system prompt is so damaging: it changes the first line, so nothing after it matches. Put variable content after the cached part. Caching also has a minimum prefix length and a limited lifetime, so it pays off for large content reused across many requests.
Not the same thing: prompt caching reuses the prompt. Caching and replaying the model's answers is response caching — rarely right for personalised conversations.
Keeping the window focused
More context is not automatically better. A 900-page manual sent with every question is expensive and buries the relevant paragraph; retrieving the sections that matter, with references, is cheaper and more accurate. In long conversations, old tool results — full order histories, search pages — keep consuming space long after they matter. Replace them with short summaries of the facts still needed.
Long tasks: compact before the limit
When a long task approaches the limit, compact it: summarise decisions, current state, open problems and next steps, then continue from that summary. Do it before the window fills — an ordinary request does not quietly drop old turns for you. Durable rules belong in files such as CLAUDE.md, not in conversation history.
Large codebases
Reading a two-million-line repository file by file fills the window long before the job is done. Search for the symbol or pattern first, read only the matching lines, or send a subagent to sweep the codebase and return just the list of locations.
Part of Domain 5: Context Management & Reliability (15% of the exam). Explanations are our own, checked against Anthropic's documentation on November 22, 2026.
Common traps
Trap
Why it fails
Do this instead
Timestamp at the top of the system prompt
Changes the prefix on every request
Move dynamic values after the cached part
Sending the whole manual every time
Costly and buries the answer
Retrieve relevant sections with references
Waiting for the window to overflow
Requests fail or get cut off
Compact into a summary before the limit
Check yourself
Two original practice questions on this lesson. Try them before opening the answers.
Check yourself 1
Scenario: Customer support resolution agent
Every request to your support agent sends the same 6,000-token system prompt and tool definitions, followed by the conversation. You want lower cost and latency. What should you do?
A. Cache the model's final answers and reuse them when a similar question arrives
B. Move the system prompt into the first user message so it is processed only once
C. Compress the system prompt with a smaller, cheaper model before each request is sent
D. Use prompt caching with a breakpoint after the unchanged tools and system prompt
Show answer and explanation
Answer: D. Prompt caching reuses an identical prefix. The prefix is evaluated in order — tools, then system, then messages — so stable content goes first, stays byte-for-byte identical, and the breakpoint sits after it (the prefix must also meet the model's minimum cacheable length). Reusing answers is response caching, wrong for personalised support. Moving text into messages does not stop it being reprocessed each request, and per-request compression adds cost.
Check yourself 2
Scenario: Customer support resolution agent
Cache hit rates are near zero even though prompt caching is enabled. The system prompt begins with "Current time: 10:41:07" followed by long static policies. What is the likely cause and fix?
A. The cache expired; send requests more often so that cache entries never time out
B. The changing timestamp breaks the prefix; move dynamic values after the cache
C. The policies are below the minimum length; pad them so they can be cached
D. Caching ignores system prompts; move the policies into the tool definitions
Show answer and explanation
Answer: B. Caching matches the prompt prefix exactly, so any change before the breakpoint — even a one-second timestamp — causes a miss on every request. Put variable content after the cached portion. Expiry and minimum length are real limits but do not explain near-zero hits with a long static prompt, and system prompts are cacheable.