Chain-of-thought prompting has demonstrated remarkable performance on various natural language reasoning tasks. However, it tends to perform poorly on tasks which requires solving problems harder than the exemplars shown in the prompts. To overcome this challenge of easy-to-hard generalization, we propose a novel prompting strategy, least-to-most prompting. The key idea in this strategy is to break down a complex problem into a series of simpler subproblems and then solve them in sequence. Solving each subproblem is facilitated by the answers to previously solved subproblems. Our experimental results on tasks related to symbolic manipulation, compositional generalization, and math reasoning reveal that least-to-most prompting is capable of generalizing to more difficult problems than those seen in the prompts. A notable finding is that when the GPT-3 code-davinci-002 model is used with least-to-most prompting, it can solve the compositional generalization benchmark SCAN in any split (including length split) with an accuracy of at least 99% using just 14 exemplars, compared to only 16% accuracy with chain-of-thought prompting. This is particularly noteworthy because neural-symbolic models in the literature that specialize in solving SCAN are trained on the entire training set containing over 15,000 examples. We have included prompts for all the tasks in the Appendix.

翻译：思维链提示法在各种自然语言推理任务中展现出卓越性能，但它在解决比提示示例更困难的问题时表现欠佳。为克服这种从易到难泛化的挑战，我们提出一种创新的提示策略——最简到繁提示法。该策略的核心思想是将复杂问题分解为一系列更简单的子问题，并按序求解，每个子问题的求解依赖于先前已解子问题的答案。我们在符号操作、组合泛化和数学推理相关任务上的实验结果表明，最简到繁提示法能够泛化到比提示中更困难的问题。值得关注的是，当GPT-3的code-davinci-002模型结合最简到繁提示法使用时，仅需14个示例即可在组合泛化基准SCAN的任何数据划分（包括长度划分）上达到至少99%的准确率，而思维链提示法的准确率仅为16%。这一突破尤为显著，因为文献中专精于解决SCAN任务的神经符号模型需在包含超过15,000个示例的完整训练集上进行训练。我们在附录中提供了所有任务的提示示例。