A moderation service classifies 400,000 short posts a day into four categories. Accuracy is acceptable but a small band of genuinely ambiguous posts is misclassified. Adding a reasoning instruction fixes most of that band in testing, but raises median response time above the service's stated ceiling. Latency is a hard constraint; accuracy on the ambiguous band is desirable. What is the best response?

The principle — When a technique helps a minority of cases and costs something on all of them, the right move is usually to change the architecture rather than the instruction. Chaining exists partly to let you spend an expensive step only where it pays.

Why the key is correct — Two pieces of guidance combine here. Anthropic states that thinking adds latency and should only be used when it will meaningfully improve answer quality, typically for problems that require multistep reasoning, advising a direct response when in doubt — which describes the bulk of this traffic. Google describes chaining as making each step a prompt, with the output of one becoming the input of the next. A first pass that classifies and reports its own uncertainty, followed by a second reasoning pass on only the uncertain slice, satisfies both: the vast majority of posts take the cheap path and stay inside the latency ceiling, while the band that actually needs deliberation gets it. It also makes the cost visible and controllable, because the proportion of traffic taking the expensive path is now a number you can measure and cap.

Why the others are wrong — Applying reasoning universally pays the cost 400,000 times to fix a small band, and treats a hard constraint as a preference. Extra examples are a reasonable cheap experiment, but examples regulate the shape of a response rather than supplying deliberation, and Google warns that too many can cause the model to overfit to them. Conditioning the reasoning inside a single prompt keeps one call but makes its latency unpredictable, which is exactly what a median-latency constraint forbids, and it hides how often the expensive branch is being taken.

Remember this — When a fix helps some inputs and costs all of them, split the path rather than the difference.

Sources — Anthropic, Prompting best practices; Google, Prompt design strategies.