Chapter 8 · Think step by step#

The big idea#

Language models write one word at a time, and each word depends on what was already written. If you ask for the final answer first, the model has no room to work. If you ask it to reason first and answer last, the earlier steps become a scratchpad that makes the final answer more reliable. This is called chain-of-thought prompting.

Everyday example: In an exam, a student who writes the working usually gets the right answer, and when they slip, you can see where. A student who jots only the answer is guessing faster. The same holds for models on maths, logic and multi-step planning.

In plain words

Ask for the reasoning, then a clearly marked final answer, so your code can pick it out: “Finish with ANSWER: ...”.

A picture#

Think step by step Asking for working lets the model check itself on multi-step problems. Think step by step Before Is 17 * 24 > 400? Answer yes/no. After Work out 17 * 24 step by step, then compare with 400, then answer. Reasoning first, answer last Same model, same question. Only the prompt changed.
Asking for working lets the model check itself on multi-step problems.

Before and after#

Replies below are typical examples to show the difference. Your model will word things differently, so run the prompts yourself.

Example 1 · Arithmetic#

Before

A shop sells pens at 12 rupees each. I buy 15 pens and pay with a 200 rupee note, then use 25% of the change to buy a sticker. How much money do I have left? Give only the number.

Typical reply

15 (a quick guess that skips a step and may be wrong)

After

A shop sells pens at 12 rupees each. I buy 15 pens and pay with a 200 rupee note, then use 25% of the change to buy a sticker. How much money do I have left?

Work it out step by step, one calculation per line, then finish with 'ANSWER: <number>'.

Typical reply

1. Pens cost 15 x 12 = 180.
2. Change = 200 - 180 = 20.
3. Sticker = 25% of 20 = 5.
4. Money left = 20 - 5 = 15.
ANSWER: 15

Prompt checker (code/promptcheck.py, columns T C A F L E D):

before   2/7  [. Y . . Y . .]  missing: task, audience, format, example, delimiters
after    3/7  [. Y . . Y . Y]  missing: task, audience, format, example

Why it works. Whether the single-number reply is right depends on luck. The step-by-step version shows its working, so you can verify it, and each line sets up the next. (Here both say 15; on harder chains the shortcut fails more often.)

Example 2 · A decision#

Before

Should we move our app to a new database? Yes or no.

Typical reply

Yes.

After

We are deciding whether to move our app from MySQL to PostgreSQL. First list the facts that favour moving, then the facts that favour staying, then weigh them. Finish with 'DECISION: move' or 'DECISION: stay' and one sentence why.

Typical reply

Favour moving: we need JSON queries and better full-text search.
Favour staying: the team knows MySQL, a migration needs about 3 weeks, nothing is currently failing.
Weighing: the new features can wait one quarter.
DECISION: stay, because the benefits are not yet worth 3 weeks of migration risk.

Prompt checker (code/promptcheck.py, columns T C A F L E D):

before   0/7  [. . . . . . .]  missing: task, context, audience, format, limits, example, delimiters
after    2/7  [. Y . Y . . .]  missing: task, audience, limits, example, delimiters

Why it works. A forced yes/no hides the trade-offs. Asking for both sides first produces an answer you can argue with.

Pull the answer out with code#

Because the prompt ends with ANSWER:, your code can find it:

reply = """1. Pens cost 15 x 12 = 180.
2. Change = 200 - 180 = 20.
3. Sticker = 25% of 20 = 5.
4. Money left = 20 - 5 = 15.
ANSWER: 15"""

answer = reply.split("ANSWER:")[-1].strip()
print(answer)

15

When not to use it. For simple lookups and rewrites, step-by-step only adds length and cost. Use it when the problem needs more than one mental step. Some modern models reason internally; asking for visible steps is still useful when you need to check the work.

Rules of thumb#

  • Ask for steps, then a final line in a fixed format.

  • Use it for maths, logic, planning and trade-offs, not for simple facts.

  • Read the steps: they reveal where the model went wrong.

Common mistakes#

  • Asking for “only the number” on a multi-step problem.

  • Trusting a long explanation just because it sounds sure; check the arithmetic.

  • Not marking the final answer, so code cannot find it.

Try it yourself#

  1. Write a prompt asking if 17 x 24 is greater than 400 with visible working.

  2. Why put the final answer last?

  3. Name a task where step-by-step is a waste.

Answers

  1. “Calculate 17 x 24 step by step, compare the result with 400, then finish with ‘ANSWER: yes’ or ‘ANSWER: no’.” (17 x 24 = 408, so yes.)

  2. Earlier tokens become working space. An answer written first cannot benefit from reasoning that comes after it.

  3. Translating a short sentence or fixing a typo. There is no multi-step reasoning to do.

Next

Longer prompts mix instructions with pasted material. Next: delimiters and structure.