Chain-of-thought concepts

What "chain of thought" means

"Chain of thought" (CoT) is the idea of getting a language model to work through intermediate steps before it commits to a final answer. The classic version is a single extra line — "Let's think step by step" — added to a prompt. The more useful modern version is asking for specific intermediate work: the quantities you extracted, the rules you applied, the cases you considered.

It helps most on problems that genuinely have multiple steps: arithmetic word problems, logic puzzles, scheduling constraints, debugging, multi-condition policy questions. It helps much less (and can just add noise and cost) on simple lookups, short rewrites, or tone changes.

Why writing steps out can help

A language model generates text one token at a time, and each new token is conditioned on everything before it. If the model jumps straight to "The answer is 42", there's no written-out intermediate result for later tokens to build on. If it first writes "Subtotal: 3 × 12 = 36, plus 6 shipping = 42", each step becomes context for the next. That's a practical, mechanical reason step-by-step prompting often improves results on multi-step tasks — not magic, and not a guarantee.

The second prompt doesn't just say "think harder". It names the intermediate values, so you can check each one: 9:40 + 2:35 = 12:15, + 25 min = 12:40, + 15 min = 12:55. Yes, with 5 minutes to spare.

The important caveat: visible reasoning is generated text

It's tempting to read a model's step-by-step explanation as a transcript of how it "really" thought. Be careful. The written reasoning is output the model generated, produced by the same process as the final answer. Research and everyday experience both show that:

• A model can write plausible-looking steps and still reach a wrong answer.
• A model can reach a correct answer while the stated steps don't actually justify it.
• The explanation can be a post-hoc rationalisation rather than the cause of the answer.

So treat visible reasoning as useful evidence you can audit, not as a guaranteed faithful trace of internal computation.

Models that reason internally

Some newer models are built to do extended reasoning before they respond. Depending on the product or API, that reasoning may be hidden entirely, shown only as a summary, or exposed in a separate field. You usually can't force a full raw transcript out of these models, and prompting them with "reveal all your hidden thoughts" tends to produce yet another generated explanation, not privileged access.

For these models, piling on "think step by step" often adds little. What still helps:

• A clear statement of the problem and constraints.
• A request for a specific, checkable output format (the key figures, the assumptions, the final answer).
• Any settings the provider offers for reasoning effort or budget, used deliberately rather than by default.

Ask for checkable work, not "your thoughts"

The most practical shift in this lesson is from "tell me what you're thinking" to "show me the pieces I need to verify you".

The improved prompt produces intermediate work that maps to things a reviewer can verify: quoted rules, amounts, per-item decisions. If the model misquotes a rule, you'll see it.

A developer-flavoured pattern

In an application, you often want the reasoning separated from the answer so your code can parse the result:

Solve the problem in <problem>. Write your working inside <work> tags, then put only the final numeric answer inside <answer> tags.

<problem>
A subscription costs $14/month or $150/year. Over 18 months, which is cheaper if the annual plan can't be prorated?
</problem>

Your code reads <answer>; a human (or a test) can inspect <work> when something looks off. Note that you're paying for the extra tokens either way, so use this where accuracy matters.

When to skip it

• Simple factual or formatting tasks — extra steps add length, latency and cost.
• When the output must be short and user-facing — keep the work in a separate section, or a separate call.
• When you're using a reasoning model that already deliberates internally — focus on a precise task description instead.