011 min
The problem Chain of Thought solves
Picture a school cafeteria that had 23 apples, used 20 for lunch and then bought 6 more. How many apples does it have now? A person solves this in two small steps. Twenty-three minus 20 leaves 3, and 3 plus 6 makes 9.
Now ask a large language model the same question and tell it to reply with the number only. The model has to go from the question to the answer in one move. It has no place to keep the 3 while it works out what comes next. Every extra stage in a problem makes that single move harder.
Word problems with several stages were a clear place where this way of using a model failed. Chain of thought is the fix. It lets the model write the 3 down before it needs it.
022 min
Where Chain of Thought comes from
The idea of letting a model write out its working has an earlier form. In 2021, Maxwell Nye and colleagues trained models to write intermediate computations into a scratchpad, which is a space in the output for working that later steps can read. They found that scratchpads improved models on tasks with many steps, from long addition to running whole programs.
In 2022, Jason Wei and colleagues at Google Research moved the idea into prompting, so no new training was needed. Their prompt held a few example questions, each followed by a worked solution in plain language. A new question went after the examples, and the model carried on in the same style. The team called the worked solution a chain of thought, meaning a series of intermediate reasoning steps. The cafeteria question above is one of the examples in their paper.
They tested arithmetic, commonsense and symbolic reasoning. The headline result used GSM8K, a benchmark of grade-school math word problems. With just eight examples in the prompt, a 540-billion-parameter model called PaLM reached state-of-the-art accuracy on GSM8K, ahead of a fine-tuned GPT-3 model that used a separate checker. The paper was presented at NeurIPS 2022.
Two meanings of the same words
In everyday speech, a chain of thought or train of thought is the flow of a person's own thinking. In AI, the phrase names a prompting technique and also the text a model writes while it works. Both are about steps, but the AI version is written output that anyone can read. It may not match what happens inside the model, and the later sections come back to that gap.
032 min
How chain of thought works
A language model writes one small piece of text at a time. Each piece is called a token, and a token is usually a word or part of a word. Every new token is chosen using everything written so far, including the model's own earlier output. Chain of thought makes use of that fact.
Go back to the cafeteria. Once the model has written that 23 minus 20 is 3, the 3 is on the page and the next step can use it. When the model reaches the 6 extra apples, it only has to add 6 to a number it can see. Each step is small enough to get right, and the final answer rests on steps that already exist.
The order matters. Wei's team ran a test in which the steps were given only after the answer, and the scores stayed close to those of plain prompting. The steps help when the model writes them before it commits to an answer.
The few-shot form
The original method is few-shot, which means the prompt contains a handful of solved examples (see few-shot learning). Each example shows a question, the steps in ordinary sentences, and the answer. The model copies that pattern for the new question. Writing good examples by hand takes effort, and the results can change when their wording or variety changes.
The zero-shot form
A shorter instruction can replace the examples. A sentence such as Let's think step by step makes many models write steps on their own. The next section gives the numbers from the test that showed it.
041 min
A concrete example: one sentence, two scores
The clearest numbers come from the zero-shot test of 2022. Takeshi Kojima and colleagues asked a model the same questions twice. The first time they asked for the answer directly. The second time they added the sentence Let's think step by step. The model was text-davinci-002, an InstructGPT model from OpenAI.
On MultiArith, a set of arithmetic word problems, the share of correct answers rose from 17.7% to 78.7%, and on GSM8K it rose from 10.4% to 40.7%.
The same paper reports that carefully written worked examples did better still on the same tasks. So the single sentence is a cheap place to start, and not the strongest form of the method.
052 min
Chain of thought variants and what each adds
A single chain can go wrong, and much of the later work on chain of thought is about that. Each variant below changes one thing: how many chains the model writes, how they are built, or what they draw on.
Self-consistency
Xuezhi Wang and colleagues asked the model for many chains on the same question instead of one. Different chains often take different routes, so they kept the final answer that most chains reached. In their tests, this raised the score on GSM8K by 17.9% compared with a single chain. The price is that you pay for every chain.
Tree of thoughts
Shunyu Yao and colleagues let the model explore branches. It proposes several possible next steps, judges which look promising, and continues from those. On Game of 24, a puzzle that asks for 24 to be made from four numbers with arithmetic, GPT-4 with chain-of-thought prompting solved 4% of tasks, and their method solved 74%. A tree needs far more model calls than a single line of steps.
Other named variants
- Automatic chain of thought writes the examples without hand work. It groups questions into clusters, picks one question from each cluster, and builds its chain with the zero-shot sentence.
- Least-to-most prompting splits a hard question into easier sub-questions and answers them in order, so each answer helps with the next.
- Multimodal chain of thought lets the steps draw on images as well as text.
- Step-back prompting first asks a more general question, then answers the original question with that answer in hand.
- Analogical prompting asks the model to recall similar solved problems before it starts.
- Contrastive chain of thought shows a correct chain and an incorrect chain in the examples.
- Tabular chain of thought lays the steps out in a table.
- Thread of thought works through a long, messy context in parts.
- Faithful chain of thought turns the steps into a program and lets a symbolic solver run it.
- ReAct mixes written reasoning with actions such as a search, so the model can check facts outside its own text.
- Retrieval-augmented generation supplies the facts from documents, and the steps then use them.
- Interactive prompting lets a person read each step and correct it before the model continues.
061 min
Chain of thought inside reasoning models
Chain of thought began as something a user asks for. Newer models are trained to do it without being asked. These reasoning models write a long chain of steps before they show an answer, which means they spend extra computation at answer time. Some products keep that text out of view, so the model's hidden thinking tokens are never shown to the reader.
A 2025 paper on DeepSeek-R1 reported that the reasoning ability of large language models can be built through reinforcement learning alone, with no human-labeled reasoning examples. Reinforcement learning is training that rewards a model for good outcomes.
Products differ in what they show. Some show the whole chain, some show a short summary, and some show nothing. As of October 2026, check what each product does before you design around the text. With a reasoning model, an instruction to think step by step may add little and may only lengthen the reply, so test it on your own task.
072 min
When chain of thought is the wrong tool
Chain of thought is not free, and it does not help everywhere. In 2024, Zayne Sprague and colleagues compared results across many papers with their own tests on 20 datasets. They report that the benefit is mostly in math and logic.
A sign that it will help
On MMLU, a broad exam of multiple-choice questions, answering directly was almost as accurate as writing steps. The exception was a question or answer that contained an equals sign, which marks symbolic work such as algebra. A rough test follows from this. If solving the task needs calculation, formal logic or several facts chained together, steps are likely to help. If the answer is a single recalled fact, steps add words and not accuracy.
Exact calculation
The same group found that much of the gain comes from carrying out symbolic steps, yet a symbolic solver still did better than the chain. A symbolic solver is a program that computes exactly. When a number must be exact, let the model call a calculator or write code, and keep the chain for the planning around it. They also found that chain of thought can be used selectively, keeping accuracy while cutting the cost of running the model.
Small models
In the original tests, size mattered. The Google team reported that chain-of-thought prompting did not help small models and gave gains only with models of around 100 billion parameters. Treat that as a 2022 finding about the models of that time, and not as a rule for today's smaller models.
Cost and delay
Every step is text that the model must generate. The team that pays for the tokens carries the money cost, and the person waiting for the reply carries the delay. A longer reply also makes the answer harder to find.
A wrong step carries forward
Each step is built from the steps before it, so an early mistake travels to the end. The chain can read smoothly while it does. A fluent chain is not evidence of a correct answer, and this is one way a model produces a hallucination that looks well argued.
082 min
Does the written chain show how the model decided?
It is tempting to read a chain of thought as a transcript of what the model was thinking. Researchers have tested that idea, and the answer is that the chain does not always show the real reason.
In 2023, Miles Turpin and colleagues designed an experiment on this. In one test, the team changed the examples so that the right answer was always option (A). The model was pushed toward (A) on new questions, and its written steps gave reasons that never mentioned the pattern. When the team pushed models toward wrong answers in this way, the written steps often gave reasons for those answers, and accuracy fell by as much as 36% across 13 tasks. The tests used GPT-3.5 and Claude 1.0.
Newer reasoning models
In 2025, Yanda Chen and colleagues planted hints in prompts and checked whether the chains admitted using them. In most settings, the chains revealed that the model had used a hint in at least 1% of the cases where it did, but the rate was often below 20%.
Reading the chain correctly
A chain is output the model produced, like any other output. It can read as a tidy explanation and still leave out what moved the answer. Treat it as working to inspect, which can help you find a wrong step, and not as proof of how the system decided. This is also why chain of thought does not replace methods that look inside the model, covered under explainability.
092 min
What chain of thought means for your work
Say a team builds an assistant that works out prorated refunds. A customer on a $120 yearly plan cancels after 4 months, so the refund covers the 8 unused months out of 12, which is $80. These figures are made up. The task has three stages and a wrong answer costs real money, so it suits chain of thought. The same approach suits code debugging, tutoring and financial calculations. Reading the steps also shows which one went wrong.
Each role makes a different decision.
- An engineer tests the assistant with and without steps on a set of the team's own refund cases. They keep the steps only if accuracy rises enough to pay for the extra tokens. Because the gains sit in symbolic work, they also move the final arithmetic into code or a calculator tool, and ask for the answer on its own marked line so a program can read it without parsing the steps.
- A product manager treats the chain as a cost with two parts: the model bill, and the seconds a customer waits. They decide which request types get the long form and which get a direct answer.
- A designer decides whether the customer sees the steps. Showing them can help someone spot a mistake. The faithfulness results mean the steps should be labelled as working, and never as the reason the system decided.
- A founder choosing a model decides whether to pay for a reasoning model that writes chains on its own, or to prompt a cheaper model for steps. The fair way to decide is to run both on the same cases and compare accuracy, cost and delay.
101 min
Chain of thought vs nearby concepts
Chain of thought is easy to mix up with other ways of breaking work into steps. The deciding question is where the steps live: inside one model reply, or across several separate calls.
| Concept | Where the steps live | Who builds the steps |
|---|---|---|
| This pageChain of Thought | ||
| AI & EngineeringPrompt Chaining | ||
| Not in the library yetTask Decomposition | ||
| AI & EngineeringFew-Shot Learning |
A team can use both prompt chaining and chain of thought. The chain splits the job across calls, and each call can still ask for steps. Splitting a goal into subtasks is the older idea of task decomposition, and chain of thought is one way a model does it inside a single reply. Tree of thoughts, covered above, differs from a chain in that it keeps several branches open at once.
?3 questions
Questions people ask
What phrase should I use to trigger chain of thought?
Do I need worked examples, or is an instruction enough?
Is chain of thought still needed with reasoning models?
§10 sources
Sources
Show all 10 sourcesHide sources
Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Brian Ichter, Fei Xia, Ed Chi, Quoc Le and Denny Zhou, "Chain-of-Thought Prompting Elicits Reasoning in Large Language Models", NeurIPS 2022.
Takeshi Kojima, Shixiang Shane Gu, Machel Reid, Yutaka Matsuo and Yusuke Iwasawa, "Large Language Models are Zero-Shot Reasoners", NeurIPS 2022.
Maxwell Nye et al., "Show Your Work: Scratchpads for Intermediate Computation with Language Models", 2021.
Xuezhi Wang et al., "Self-Consistency Improves Chain of Thought Reasoning in Language Models", ICLR 2023.
Shunyu Yao et al., "Tree of Thoughts: Deliberate Problem Solving with Large Language Models", NeurIPS 2023.
Miles Turpin, Julian Michael, Ethan Perez and Samuel R. Bowman, "Language Models Don't Always Say What They Think: Unfaithful Explanations in Chain-of-Thought Prompting", NeurIPS 2023.
Yanda Chen et al., "Reasoning Models Don't Always Say What They Think", 2025.
Zayne Sprague et al., "To CoT or not to CoT? Chain-of-thought helps mainly on math and symbolic reasoning", ICLR 2025.
DeepSeek-AI, "DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning", 2025.
DAIR.AI, "Chain-of-Thought Prompting", Prompt Engineering Guide (an overview of the variants).





