Sign in Subscribe
news

Claude Code Prompt Caching

Sebastien Dubois

Sebastien Dubois

— 4 min read

Canonical version: Claude Code Prompt Caching.

Claude Code uses Anthropic's prompt cache to reuse large, stable chunks of context (system prompt, tool definitions, CLAUDE.md, prior turns) across requests instead of resending and re-billing them every turn. Cache reads are cheaper than uncached tokens; cache writes are more expensive than normal tokens. Net savings depend on your usage pattern — context window size, whether the request is the main agent or a subagent, how often a session resumes, etc.

5-minute vs 1-hour cache

Anthropic supports two cache durations. The trade-off is simple: a longer TTL wins when the same cached prefix is reused many times within the window; a longer TTL loses when you pay the higher write price and then never hit the cache again.

  • 5m cache: good for short, interactive bursts. Pays off if the session keeps sending requests with the same prefix within a few minutes.
  • 1h cache: good for long-running or returning sessions. Write is more expensive, so it only pays off when the cached prefix is actually reused during the hour. A single-shot agent invocation with 1h caching is a worst case — you pay the higher write cost and throw the cache away.

Defaults in Claude Code

Claude Code picks the duration per query type, not globally, based on expected reuse patterns:

  • Subscribers (Pro/Max, logged-in CLI): 1h cache is rolled out as the default in several places where reuse is likely. Subagents and other rarely-resumed queries stay on 5m on purpose, to avoid paying the 1h write premium for caches that will never be reused.
  • API customers: still default to 5m everywhere. Anthropic wants more real-world data before flipping 1h on by default for API usage.
  • Coming soon: client-side default is moving to 1h for a few more query types, plus environment variables to force 1h or 5m explicitly. Useful for users who know their workflow better than the heuristic does.

Expected token savings from these heuristics are small — not 12x or anything dramatic. It is a net win on average for the targeted query types, not a step change.

Telemetry interaction

The 1h/5m decision is driven by experiment gates cached client-side. Turning off telemetry (DISABLE_TELEMETRY=1, see Claude Code Configuration) also disables experiment gates — Claude Code does not phone home when telemetry is off — so the client falls back to the hard-coded default, which today is 5m for all query types. If you care about the 1h optimization, keep telemetry enabled. If you care about not phoning home, expect to pay the uncached-prefix cost on queries where 1h would have helped.

Idle compaction

The 1h TTL leaves one expensive case: you step away from a long session, the cache expires, and your next message reprocesses the whole conversation without any cache hit. On a session with several hundred thousand tokens of history, a simple "ok, continue" after lunch costs you that entire prefix again.

Idle compaction is Claude Code's answer to that case. Lydia Hallie (Claude Code team) announced it on 2026-10-14, and the changelog lists it under v2.1.299. When a long conversation has been idle for about 54 minutes, Claude Code compacts it shortly before the one-hour cache expires. Your next message then sends the summary instead of the full history. When you come back, the conversation shows Compacted while idle, before the prompt cache expired.

It's on by default, but it only fires when all of these conditions hold:

  • You're in an interactive terminal session
  • The conversation is on the one-hour prompt cache TTL (so nothing happens on 5m caches)
  • Claude Code can fetch feature flags from Anthropic. DISABLE_TELEMETRY, DO_NOT_TRACK, CLAUDE_CODE_DISABLE_NONESSENTIAL_TRAFFIC and third-party providers (Bedrock, Foundry, etc.) all skip that fetch, so idle compaction never runs there
  • The context has reached the threshold: 200,000 tokens by default
  • About 54 minutes have passed since the last request, and your computer is awake at that point

The 200K default effectively targets 1M-context sessions. On a 200K window, regular auto-compaction kicks in before the conversation ever reaches 200,000 tokens.

Like any compaction, it has a cost. You save usage, but Claude continues from a summary instead of your full context. Whatever the summary drops (a failed approach, an exact error message, a constraint you mentioned once) is gone for the rest of the session. For a long debugging or design session where the nuance matters, paying to reprocess the full context is often worth it. The other option is to run /compact focus on <what matters> yourself before stepping away, so you decide what the summary keeps.

How to control it (details in Claude Code Configuration):

  • "idleCompaction": false in any settings file turns it off (needs v2.1.290 or later; on earlier versions, only turning off autoCompactEnabled stops it)
  • CLAUDE_CODE_IDLE_COMPACT_MIN_TOKENS changes the threshold. Values below 100,000 are raised to 100,000. Lower it (e.g., 150000) if you want 200K-window sessions to qualify, raise it if you only want it on huge sessions
  • DISABLE_AUTO_COMPACT=1 turns it off for one session, together with auto-compaction (/compact keeps working)

Why the nuance matters

Prompt caching is one of the few cost knobs users can actually tune in an agent harness. The same cache duration can be a win, a wash, or a net loss depending on workflow. Main-agent loops with long context windows and many turns benefit most. One-shot subagent calls, short sessions, or workflows that rotate through fresh contexts get less (or negative) value from a longer TTL. Anthropic is rolling 1h out selectively rather than globally for this reason.

Related mechanism at the model level: AI KV Cache (attention-layer cache for generation, different from prompt caching which deduplicates input tokens across requests).

References

About Sébastien

Ready to get to the next level?

Found this valuable? Share it with someone who needs it.

Join 6,000+ readers. Get practical systems for knowledge & AI. Free.

Subscribe ✨

Free: Knowledge System Checklist

A clear roadmap to building your own knowledge system. Subscribe and get it straight to your inbox.

6,000+ readers. No spam. Unsubscribe anytime.

Subscribe