Where you edit your prompt decides what it costs

Twenty turns into a session the answers have been arriving in a couple of seconds. Then you add three lines to the top of your project instructions, or flip the model in the dropdown, or come back from a coffee — and the next reply takes half a minute while the usage bar moves further than it has all morning. The question was not harder. Nothing you can see about the conversation changed.

That reads as "the model got slower today". It is not that. It is a cache miss, and which of your actions caused it had almost nothing to do with how much you changed. It had to do with where.

The vendors document the machinery in detail — cache breakpoints, minimum cacheable lengths, time-to-live, what a write costs against what a read costs — all of it in the register of configuration, for someone building an application. What none of it draws out is the consequence for everybody else, including a person who has never touched an API. That consequence is an ordering rule, and position, not quantity, is the variable.

Here is the machine, then the rule.

The unit being reused is a prefix

A model remembers nothing between requests. Every turn, the whole thing is assembled and sent again — instructions, tool definitions, project files, every previous message and tool result, and then your new line at the end. Anthropic's cost guidance states the shape plainly: "Claude Code sends your full conversation with every request, and each time Claude uses tools it sends another request carrying that batch of tool results." What that re-sending costs you in usage is a separate article's subject; here it only matters that the request is mostly a copy of the last one.

Caching is how a provider avoids re-processing the copied part. It stores the processed form of a prefix — everything from the start of the request up to some point — under a hash of exactly that text. Next request, it hashes your prefix and looks for the same entry.

So the reusable part is not "the files you did not touch" and not "the messages you did not edit". It is a contiguous run of text from the beginning. Anthropic's Claude Code documentation says the consequence in one line: "The match is exact, so a change anywhere in the prefix recomputes everything after it. There is no per-file or per-segment caching."

Read that as geometry, not policy. Everything before your edit is reusable. Everything from your edit onward is not.

Exact means exact, and the hash is cumulative

The word "exact" is doing more work than it looks like. The API documentation is unambiguous: "Cache hits require 100% identical prompt segments, including all text and images up to and including the block marked with cache control."

Identical includes what you do not think of as prompt text. Tool definitions — names, descriptions, parameter schemas and their order. Whether an image is present at all. Settings rendered into the request rather than sent beside it. The hash covers the whole run, so one differing character makes everything from that character onward a miss, and the size of the difference is irrelevant to the size of the bill.

Hence the confusing fingerprint: nothing you care about changed, and the entire context was reprocessed.

What sits in front of what

Because the reusable region always starts at the beginning, the only thing that decides an edit's blast radius is how much material sits behind it. Both major providers order the request the same way, and Anthropic states the hierarchy directly: "The cache follows a prefix hierarchy (tools → system → messages), so a change at one level invalidates that level and everything after it"

In an agent like Claude Code that hierarchy has names you recognise. The system prompt layer holds core instructions, tool definitions and the output style. The project-context layer holds CLAUDE.md, memory files and unscoped rules, read once at launch: "CLAUDE.md files are loaded into the context window at the start of every session, consuming tokens alongside your conversation." The conversation layer holds your messages, the replies and every tool result.

A new message touches only the last layer, so the first two stay cached. A change to the first invalidates all three, because everything else now sits behind a different prefix. Which layer a given instruction belongs in is its own decision; what this page adds is that the decision has a price attached, and the price is everything downstream of it.

The list of things that quietly recompute everything

Each of these produces one slow, expensive turn, after which the new prefix is cached and speed returns. That recovery is what makes them hard to attribute.

Switching the model. Each model has its own cache. The content is byte-for-byte identical and the next request still reads none of it. Fingerprint: you swap models to "check something quickly" and the check costs more than the work.

Changing the effort or thinking setting. The configuration is rendered into the request, so it is part of the key. Fingerprint: you raise effort for one hard question and the whole conversation is reprocessed to ask it.

Toggling a connector or MCP server. Where an agent defers tool definitions and appends them on demand, a server appearing or vanishing costs nothing; where they load upfront, any change to the set invalidates from the tools layer down — and a server can connect or drop without you doing anything. Fingerprint: an unattended turn is inexplicably slow.

Reordering or re-describing tools. Not adding one. Reordering. The prefix is text, and text has an order. Fingerprint: a config tidy-up you would swear was cosmetic.

Editing the top of a system prompt or a project file. One line at the top invalidates every token after it, including the entire conversation. One line at the bottom invalidates only itself.

Compacting. Summarising the history replaces it, so the new history shares no prefix with the old. Often worth it — but a rebuild, not a free tidy.

Upgrading the tool. A new version usually changes the system prompt or the tool definitions, so the first request after it starts from the top. Resuming a long old session after an upgrade can be the most expensive request you send.

The rule, stated as a rule

Sort your material by how often it changes. Most stable first. Most volatile last.

OpenAI's caching guide gives the same instruction as a best practice: "To realize caching benefits, place static content like instructions and examples at the beginning of your prompt, and put variable content, such as user-specific information, at the end." Its latency guide repeats it as a speed technique rather than a cost one: "Maximize shared prompt prefix, by putting dynamic portions (for example, RAG results and history) later in the prompt."

The generalised form, which no page states because no page is addressing you: the cost of a change is the number of tokens that follow it. Not the number of tokens in it. That reframing is what turns a pile of documented parameters into a habit you can hold.

A date near the top is the most expensive character you can type

The worst thing to put at the front of a stable block is anything that varies per request, and the purest example is a clock. "Today's date is …". A session ID. A "last updated" line you keep current out of tidiness.

Both vendors document the same trap with the same diagram. A request shaped stable instructions → timestamp → your message writes a fresh cache entry every single time and never reads one, because yesterday's entry was keyed on yesterday's timestamp. You pay the write premium on every request and collect the read discount on none. The fix is not to delete the date but to move it behind everything stable — to the end of the prefix, or out of the prompt entirely into metadata the model never sees.

Cache-write tokens are priced above ordinary input and cache reads well below it (Checked 23 August 2026 — treat any such multiplier as a figure that moves.), so a prefix that always writes and never reads is strictly worse than not caching at all.

Three habits that look harmless

Tidying the top of your instructions file mid-task. The edit is small, the file is small, and the invalidation is the whole conversation behind it. Worse, in some agents the edit does not apply until the next restart — so depending on the tool you pay everything, or you pay nothing and gain nothing. Append at the bottom while you work; reorganise at the top when you are about to restart anyway.

Toggling connectors as you go. Switching a server off to "save context" mid-session can cost more than leaving it on, because the saving is a few hundred tokens per turn and the cost is one full reprocess.

Grazing the model and effort dials. Both are part of the cache key. Two switches in a long session are two full reprocesses. Choose at the top of a session, and change deliberately.

A pause is a cost, because the cache runs on a clock

Cached prefixes expire after a period of inactivity, and every request that reads one resets the timer. The default lifetime is short — five minutes on the API, with a one-hour option that bills writes higher, and agents on a subscription typically ask for the longer one (Checked 23 August 2026 — TTLs and which tier gets which are exactly the figures that move.)

Two consequences most people never connect. The clock starts when the request starts, not when the reply finishes, so a long generation eats its own window. And your first message after any real break misses and reprocesses the entire accumulated context at full price — which is why the first question of the afternoon is the slowest of the day and the one after it is fine.

There is a floor too: prompts below a minimum length are not cached at all, and no error tells you so.

Telling a cache miss from a slow model and a long answer

Three different problems produce "it felt slow", and they have distinct signatures.

A cache miss is a long silence before the first token, then normal streaming — and it happens exactly once, on the turn after something changed or after a gap. If the next turn is fast again, it was the cache.

A slow model or a high effort setting delays the first token on every turn, not one. If turn after turn pauses identically, the setting is the cause, not the prefix.

A long answer starts promptly and simply keeps going. The delay is in the streaming, not before it.

If you want the fact rather than the feeling, the API reports it on every response: tokens read from cache against tokens written to it. A high read-to-write ratio means it is working. Writes staying high turn after turn mean something in your prefix keeps changing and you have not noticed what.

What this changes on your meter, and what it does not

Caching changes the price of re-reading your context. It does not change the size of it. Cached tokens still occupy the context window and still count toward rate limits, so a cheap conversation and a small one are different things, and only the second is a fix.

It also does not touch the answer. A reply after a miss is not more careful and a reply after a hit is not degraded. If an answer changed after you edited the top of a file, the cache is not why — that is sampling, and it would have changed anyway.

One boundary looks like a bug and is not: a subagent starts from its own system prompt and tool set, so its first request shares no prefix with its parent and reads none of the parent's cache. What actually crosses that boundary is a separate subject from what it costs.

The decision list, for a case this article never described

1. Ask where it changed, not how much. The bill is the tokens after the edit. A one-word change at the top is expensive; a thousand words appended at the bottom are cheap.
2. Order everything you control by how often it changes. Instructions and definitions first, reference material next, per-request material last. That applies to a prompt, a project file and a document you keep pasting in.
3. Keep clocks, IDs and anything auto-generated out of the front. If a value varies per request and the model does not need it, it belongs in metadata.
4. Make settings decisions once, at the start. Model, effort, connectors, plugins — every mid-session change is a full reprocess.
5. Append; do not rewrite. Editing or reordering earlier content is the one operation with no cheap version.
6. If you must invalidate, invalidate once. Batch every front-of-prompt change into a single restart.
7. Treat a long break as the end of a session. Past the lifetime there is nothing left to reuse, so resuming a huge thread costs more than starting a focused one.
8. Measure before believing any of this about your own setup. The two cache counters are the only evidence; everything above is a prediction they can falsify.

Product names, TTLs and minimum lengths will have moved by the time you need this. The geometry will not. A cache is keyed on an exact prefix, and the question that decides your latency and your bill is not what you edited — it is how much text was standing behind it.