Chain-of-Thought Prompting

Chain-of-thought (CoT) prompting is a technique in prompt engineering that elicits step-by-step reasoning from a large language model by including intermediate reasoning steps—the "chain of thought"—in the prompt or by instructing the model to generate such steps before producing a final answer. Rather than asking a model to jump directly from a question to a conclusion, CoT prompting encourages it to decompose the problem and work through it sequentially, mirroring the way a human might reason through a mathematics problem or a logical puzzle. The technique was formalised and popularised by Jason Wei and colleagues at Google Brain in a 2022 paper demonstrating that simply providing a few examples that included reasoning steps significantly improved the performance of large language models on arithmetic, commonsense, and symbolic reasoning benchmarks.^[1] The improvement was particularly pronounced in larger models, suggesting an emergent capability that scales with model size.

How Chain-of-Thought Prompting Works

In a standard prompt, a model receives a question and is expected to produce a direct answer. In a chain-of-thought prompt, the model is additionally shown—or instructed to produce—a sequence of reasoning steps that lead logically to the answer. For example, a prompt might read: "Q: Roger has 5 tennis balls. He buys 2 more cans of tennis balls. Each can has 3 balls. How many tennis balls does he have now? A: Roger started with 5 balls. 2 cans × 3 balls = 6 balls. 5 + 6 = 11 balls." By providing this structured reasoning in the context, the model learns the expected format and applies analogous reasoning to new problems.

Variants

Zero-Shot CoT

The simplest variant requires no examples at all. Appending the phrase "Let's think step by step" to the end of a question prompts many large language models to spontaneously produce intermediate reasoning. This zero-shot approach was identified by Kojima et al. (2022) and works because instruction-tuned models have been trained on so many examples of step-by-step explanation that the phrase activates a corresponding generation mode.^[2]

Few-Shot CoT

In few-shot CoT, the prompt includes several complete worked examples, each showing both the reasoning chain and the final answer. This is especially useful when the desired reasoning style is specific—for instance, structured mathematical proofs, debugging traces, or multi-step logical deductions—and when the task is sufficiently complex that zero-shot prompting fails to produce the right structure.

Auto-CoT

Auto-CoT automates the construction of few-shot examples by sampling questions from a task dataset, clustering them by semantic diversity, and generating reasoning chains for each cluster using zero-shot CoT. The resulting demonstrations are more diverse and less likely to share common errors than manually curated examples.^[3]

Multimodal CoT

Introduced by Zhang et al. (2023), multimodal CoT extends the technique to settings where reasoning draws on both language and visual inputs—for example, solving a science question that requires interpreting a diagram alongside textual information. The model first generates a rationale from both modalities and then produces its answer conditioned on that rationale.^[3]

Relationship to Reasoning Models

Chain-of-thought prompting has directly influenced the design of a new class of reasoning models that generate extended internal thinking before producing output. OpenAI's o1 and o3 series, Anthropic's Claude "extended thinking" mode, and DeepSeek-R1 all implement variants of chain-of-thought reasoning at training time through reinforcement learning, so the model learns to produce reasoning traces as an intrinsic behaviour rather than requiring explicit prompting.^[4] DeepSeek-R1, released in January 2025, uses a self-supervised mechanism to refine its reasoning chains, achieving strong performance on mathematical and coding benchmarks.

Limitations

Chain-of-thought prompting improves reasoning accuracy but does not eliminate errors. Models can produce plausible-sounding but incorrect reasoning chains—a form of reasoning hallucination in which each step sounds locally coherent but the global conclusion is wrong. CoT also increases the length of model outputs, increasing inference cost and latency. For very long reasoning chains, models may lose track of intermediate results, especially at shorter context windows. Chain-of-thought prompting has become a practical concern for Malaysian enterprises deploying LLMs, particularly in domains that require verifiable reasoning. Malaysian law firms exploring AI-assisted contract review have found that CoT-structured prompts produce more auditable analysis: the model explains which clauses it considered and why it flagged them, making the output easier for lawyers to verify and challenge. This aligns with the Malaysia AI Governance Framework emphasis on transparency and explainability for high-risk AI applications. In financial advisory services, Malaysian fintech companies regulated by the Securities Commission Malaysia (SC) have explored CoT prompting to generate structured investment rationale—walking through risk factors, regulatory considerations, and market conditions before recommending a course of action. The step-by-step format makes it easier for compliance officers to review AI-generated advice against SC guidelines. Malaysian universities, including Universiti Malaya and Universiti Kebangsaan Malaysia (UKM), have incorporated CoT prompting into AI literacy curricula, using it as a pedagogical tool to demonstrate how LLMs reason and where they can go wrong. This supports Malaysia's broader AI talent development goals under the MyDigital Blueprint, which targets producing 20,000 AI professionals by 2025. From a Bahasa Malaysia perspective, CoT prompting has been found to particularly benefit reasoning tasks in the Malay language, where the relative scarcity of Malay training data means that models benefit more from explicit structuring of reasoning steps. Local AI researchers at UTM and UPM (Universiti Putra Malaysia) have documented improved accuracy on Malay-language mathematics and logical reasoning benchmarks when zero-shot and few-shot CoT prompts are applied to multilingual models.

See Also

References

1. Wei, J., Wang, X., Schuurmans, D., et al. (2022). Chain-of-Thought Prompting Elicits Reasoning in Large Language Models. Advances in Neural Information Processing Systems, 35. arXiv:2201.11903.
2. Kojima, T., Gu, S. S., Reid, M., Matsuo, Y., & Iwasawa, Y. (2022). Large Language Models are Zero-Shot Reasoners. arXiv:2205.11916.
3. Zhang, Z., Zhang, A., Li, M., et al. (2023). Multimodal Chain-of-Thought Reasoning in Language Models. arXiv:2302.00923.
4. Promptingguide.ai. (2024). Chain-of-Thought Prompting. https://www.promptingguide.ai/techniques/cot