Skip to content

Technique

Advanced

Self-consistency

Generate several independent reasoning paths for the same problem and keep the answer they most often agree on.

What it is

Self-consistency is a technique for improving answers on problems with a single correct result, such as math, logic, or classification. Instead of trusting one response, you sample multiple independent reasoning paths for the same question, then pick the most common final answer (a majority vote).

The idea: a correct answer can usually be reached in several ways, while errors tend to scatter in different directions. Agreement across independent attempts is evidence, not proof, that the answer is right.

When to use it

  • Math, logic, or estimation problems with one correct answer
  • Classification decisions where an occasional error is costly
  • Checking how stable an answer is before relying on it
  • When accuracy matters more than speed or cost

How it works

The procedure

  1. Write a prompt that asks the model to reason through the problem and finish with a clearly marked final answer.
  2. Run it several times independently (for example 5 times). Answers need to vary, so the runs must use some randomness: separate fresh chats in an app, or a non-zero temperature via an API.
  3. Extract the final answer from each run.
  4. Take the majority (or most consistent) answer.
Prompt
A store sells notebooks at 3 for $4. Maya needs 14 notebooks for her class.
The store only sells them in packs of 3. How much will she spend?

Think it through step by step, then end with one line:
FINAL ANSWER: $<amount>

Illustrative results from five independent runs:

Illustrative example response — written by Promptly, not generated live
Run 1: FINAL ANSWER: $20 Run 2: FINAL ANSWER: $20 Run 3: FINAL ANSWER: $18.67 Run 4: FINAL ANSWER: $20 Run 5: FINAL ANSWER: $20

Four of five agree on $20 (5 packs × $4, since 14 notebooks need 5 packs). The outlier ignored the packs-only rule. The vote filters out that one-off mistake.

Important details

  • Answers must be comparable. Voting works when final answers are short and well-defined (a number, a label, a choice). For open-ended writing there's nothing to count, though you can ask a model to choose the most consistent of several drafts.
  • Independence matters. Asking "are you sure?" in the same chat isn't the same thing; the model sees its earlier answer and may just repeat or flip it.
  • Agreement isn't proof. If the model has a systematic misunderstanding, every run can be wrong in the same way.

Watch out: Self-consistency multiplies cost and time. Five samples cost roughly five times as much as one. Use it for important, checkable questions, not everyday drafting.

Note: The written step-by-step reasoning helps the model work through multi-step problems, but it's generated text, not a guaranteed window into how the model reached its answer. Judge runs by their final answers, and verify critical results independently.

Step by step

  1. 1

    Write a prompt that asks for reasoning and a clearly marked final answer in a fixed format.

  2. 2

    Run it several times independently (fresh chats, or sampling with some randomness via an API).

  3. 3

    Extract the final answer from each run.

  4. 4

    Count the answers and take the majority.

  5. 5

    If there's no clear majority, treat the question as uncertain: add context, simplify, or verify another way.

Weak vs strong

Weak prompt
Solve this scheduling puzzle. Are you sure? Check again. Are you really sure?
Why it falls short: Repeated "are you sure?" in one conversation isn't independent sampling. The model sees its prior answer and may stubbornly repeat it or flip to a wrong one under pressure.
Strong prompt
(Run 5 times, each in a new chat) Three friends split a dinner bill of $137.40 plus an 18% tip. One friend only had a $12 salad and pays just their own share plus tip. The other two split the rest evenly. How much does each of the other two pay? Reason step by step, then end with: FINAL ANSWER: $<amount per person>
Why it works: Independent runs, a problem with one correct answer, and a fixed final-answer line make the results easy to compare and vote on.

Practical use cases

  • Math and word problems

    Check homework-style calculations, budgeting math, or unit conversions by comparing several independent answers.

  • High-stakes classification

    In an automated pipeline, label each item several times and flag items without a clear majority for human review.

  • Logic and scheduling puzzles

    Solve constraint problems (who sits where, which meeting slot works) where one reasoning slip changes the answer.

  • Measuring confidence

    Use the level of agreement as a rough signal: unanimous answers are more trustworthy than a 3-2 split.

Practice exercise

Run your own vote. Pick a tricky word problem, such as: "A train leaves at 9:40 and arrives at 13:15, with a 25-minute stop. What was the total time spent moving?"

  1. Write a prompt that asks for step-by-step reasoning and ends with FINAL ANSWER:.
  2. Run it in 5 separate new chats and record each final answer.
  3. Take the majority, verify it yourself, and note whether any runs disagreed and why.
Practice in the PlaygroundAll techniques