When to use a reasoning model — and the tasks where it makes things worse

You have a dropdown with a thinking model in it and a fast one, or a slider that goes from none to maximum. Nobody has told you which task deserves which, so you pick one of two habits and keep it: maximum on everything, or never touch it. Both are wrong in the same way — they answer a question about the task with a setting about yourself.

The vendors document the machinery well. What they publish is what reasoning is, what the controls do, and what each model is good at. What almost nobody publishes is the decision — given the thing in front of you right now, does more reasoning make the answer better, make no difference, or make it worse? All three happen, and the third one surprises people.

Here is the machine, and then the rule.

What you are buying is intermediate work

A model without reasoning writes its answer straight out, one token after another, with no draft. Anthropic puts the consequence well: "A model that answers in a single pass has to get everything right on the first try: no scratch work, no checking, no changing course halfway through."

Reasoning is the scratch work. The model generates a stretch of hidden text first — restating the problem, trying an approach, noticing it fails, backing up — and then writes the answer conditioned on all of that. OpenAI describes the same thing from the other side: the reasoning tokens let the model "plan, use tools effectively, inspect alternatives, recover from ambiguity, and solve harder multi-step tasks."

Hold onto the word intermediate. You are not buying a smarter model. You are buying room for the model to be wrong in private before it commits in public. That framing is the whole decision, because it tells you exactly when there is nothing to buy.

The tokens are real, generated, billed, and then thrown away

The scratch work is not metaphorical. It is generated text that lands nowhere you can read.

Anthropic states the bill plainly: "Thinking has a cost: the tokens Claude spends reasoning are billed as output tokens, even when the thinking text isn't returned to you, and they count toward max_tokens alongside the response text." OpenAI's reasoning guide says the same about space — the hidden tokens occupy the context window like anything else, and "Depending on the problem's complexity, the models may generate anywhere from a few hundred to tens of thousands of reasoning tokens."

Three consequences follow.

The reasoning competes with your own input for room. The guide recommends "reserving at least 25,000 tokens for reasoning and outputs when you start experimenting with these models" — that is space your document, your code and your conversation do not get.

It costs you time in direct proportion, because generation is the slow part. OpenAI's latency guidance is that "Generating tokens is almost always the highest latency step when using an LLM: as a general heuristic, cutting 50% of your output tokens may cut ~50% of your latency." Reasoning tokens are generated tokens. Turning the dial up is buying seconds of silence.

And it consumes your usage allowance at the same rate as any other output — one sentence's worth of the story here, a whole article's worth in what actually uses up your Claude usage limits. This page is about whether the answer gets better; that one is about what it costs.

The test: is there a middle where you can be wrong?

Now the rule, and it is one question.

Does this task have an intermediate step where getting it wrong makes the final answer wrong?

If yes, reasoning pays, because that is precisely the step it buys. If no — if the answer is already present in the input and the work is to locate it, restate it or reshape it — then there is no middle to protect, and the reasoning tokens are spent producing deliberation about something that was never in doubt.

Notice what the test does not ask. It does not ask whether the task feels hard, or important, or long. A hundred-page contract summarised faithfully is a big job with no middle. A four-line puzzle with two interacting constraints is a small job that is nothing but middle.

Shapes with a middle

These are the ones where the dial earns its cost:

• Multi-step deduction. Any question where step three depends on step two being right. Arithmetic over facts, dependency chains, "if this changed, what else breaks".
• Constraint satisfaction. A schedule, a plan, a config, a design that has to satisfy five things at once. The failure mode of a single-pass model is satisfying the first four and quietly dropping the fifth.
• Planning and delegation. Deciding the sequence of moves before making any of them, which is what agentic work is.
• Ambiguous or under-specified requests. Where the model must first work out what you actually meant. Reasoning models tend to notice the gap rather than paper over it.
• Code with interacting parts. Not "write me a regex" — a change whose correctness depends on three files agreeing. Anthropic names this class directly: thinking helps "on complex tasks like math, coding, analysis, and long-running agentic work, where the quality of the answer depends on intermediate work that would otherwise be compressed into the response itself or skipped."
• Judging someone else's output. Grading, reviewing, and checking are deduction problems wearing a different hat.

Shapes with no middle

Here the dial buys latency and nothing else:

• Retrieval and lookup. The answer is in the text you pasted. Finding it is not deduction.
• Formatting and transformation. JSON to CSV, prose to bullets, renaming things. The output is a function of the input.
• Summarising given text. Faithful summarisation is compression, not inference. It gets worse when the model starts inferring.
• Classification with a clean taxonomy. If a competent human would answer instantly and never disagree with another competent human, there is no middle.
• Short conversational turns. Latency is the product here.
• Anything you would have hard-coded. OpenAI's latency guide makes the wider version of this point: "Language models are powerful and versatile, and are therefore sometimes used in cases where a faster classical method would be more appropriate." Before asking whether the task deserves reasoning, ask whether it deserves a model.

Where more reasoning actively hurts

"Does nothing" is the common case. "Makes it worse" is the one worth knowing about, because it does not look like a settings problem when it happens.

In an interactive loop, latency is quality. A slightly better answer that arrives after a long pause loses to a good one that arrives now, because you would have caught the flaw yourself and asked again — twice — in the same wall-clock time. Deliberation you have to wait for is only worth it when you cannot cheaply check the result.

A reasoning budget can eat the answer. If generation hits the ceiling while the model is still thinking, you get a truncated response — and OpenAI is blunt about what that means: "This might occur before any visible output tokens are produced, meaning you could incur costs for input and reasoning tokens without receiving a visible response." You paid, and there is nothing on the screen.

On summarising and extraction, inference is the bug. When the job is "tell me what this document says", a model with room to deliberate has room to reconcile, editorialise and fill gaps. That is not a better summary. That is a summary with your source's meaning bent toward the model's.

The control is a dial, and higher is not a synonym for better

The effort setting — whatever your product calls it — is documented as a trade, not an upgrade. OpenAI: "Lower effort favors speed and lower token usage, while at higher effort the model thinks more completely to provide higher quality responses." Both halves are real, so the right setting depends on which half you are short of.

Two things follow. Modern models reason adaptively — already spending less on easy inputs and more on hard ones — so the dial shifts a distribution rather than forcing a fixed amount. And a maximum setting is not the fix for a bad answer: if the request was vague, longer deliberation over the same vagueness produces a more elaborate wrong answer.

Prompting a reasoning model is a different craft

This is where habit does the most damage, because the technique everybody learned is the one that backfires.

"Let's think step by step" was a genuine improvement for single-pass models: it made them externalise reasoning they would otherwise skip. Ask a reasoning model for it and you are asking for something it already did, in the one place it cannot help — the visible output. OpenAI says it outright: "Some prompt engineering techniques, like instructing the model to "think step by step," may not enhance performance (and can sometimes hinder it)."

The same reversal applies to few-shot examples. Try zero-shot first; reasoning models often do not need examples, and a set of examples that does not exactly match your instructions gives the model a contradiction to resolve instead of a task to do.

What to write instead is short: the goal, the constraints, and the shape of the output you want — then stop. Do not prescribe the intermediate steps; that is the part you are paying it to work out. Constraints are where to be maximal ("under 400 words", "must not modify the schema", "return only the diff"), because those are what the deliberation is checked against. Where that instruction physically belongs — your message, a system prompt, a project instruction — is a separate question.

The honest limit: reasoning cannot repair the input

Every claim above is about how well the model uses what it was given. None of it touches whether what it was given was right.

OpenAI's accuracy guide splits failures along exactly this line, and the retrieval half is not a model problem at all: "You can supply the wrong context, so the model can't possibly answer, or you can supply too much irrelevant context, which drowns out the real information and causes hallucinations."

Can't possibly answer. Not "answers less well at low effort". If the fact is missing, stale, or contradicted by another document in the same prompt, no amount of thinking recovers it, and the most expensive setting will produce the most confident version of the wrong answer. Reasoning fixes the other half of the guide's split — the case where "the model can also get the right context and do the wrong thing with it."

So diagnose before you spend. If the answer is wrong because a fact was absent, fix the input. If it is wrong because the steps were mishandled, then the dial is the right tool.

Variance survives the dial too

One more thing reasoning does not buy: repeatability. A reasoning model is sampled like any other, so a run that succeeded once tells you very little. Turn the dial up, get a good answer, conclude the setting fixed it, and you have measured a single sample — why the same prompt gives you a different answer is about exactly that trap. Judge any change over several runs.

Often the answer is both, at different steps

The framing of "which model" hides the better option: stop treating the workload as one task.

Reasoning is worth its cost on the step that decides things and wasted on the steps that carry the decision out. OpenAI calls the split mainstream rather than advanced: "Most AI workflows will use a combination of both models—o-series for agentic planning and decision-making, GPT series for task execution."

It works by hand too, in a chat window, with no infrastructure. Ask the hard question at high effort, take the plan, then drop the effort and work through it. You have bought deliberation exactly where the middle was.

The rule, applied to a task you have not seen yet

The decision, in the order it is worth asking:

1. Would a classical tool or a lookup do this? If yes, no model, and certainly no reasoning.
2. Is the answer already in the input? If the work is finding, reformatting or faithfully compressing, keep reasoning low. There is no middle to protect, and inference is a risk rather than a benefit.
3. Is there a step where being wrong makes the answer wrong? If yes, raise it. Deduction, constraints, planning, ambiguity, interacting code.
4. Am I waiting on this? Interactive turns are a latency product. Deliberate asynchronously, respond fast.
5. Can I check the answer cheaply? If verification is easy — it compiles, the test passes, you can read it in ten seconds — a fast answer plus a check beats a slow answer you trust on faith.
6. Is the failure a missing fact or a mishandled step? Only the second one is the dial's to fix. The first is an input problem and no setting touches it.
7. Can I split it? Reason once on the decision, execute cheaply on the rest.

Model names will change, and so will the labels on the control. The question underneath does not: you are deciding whether this task has a middle worth protecting, and paying for privacy in which to be wrong only when it does.

And one closing check that costs nothing. If you would happily accept the first plausible-looking answer without reading it carefully, you did not need the deliberation. If you would not, that is the task the dial exists for.