Current literature, aiming to surpass the "Chain-of-Thought" approach, often resorts to an external modus operandi involving halting, modifying, and then resuming the generation process to boost Large Language Models' (LLMs) reasoning capacities. This mode escalates the number of query requests, leading to increased costs, memory, and computational overheads. Addressing this, we propose the Algorithm of Thoughts -- a novel strategy that propels LLMs through algorithmic reasoning pathways, pioneering a new mode of in-context learning. By employing algorithmic examples, we exploit the innate recurrence dynamics of LLMs, expanding their idea exploration with merely one or a few queries. Our technique outperforms earlier single-query methods and stands on par with a recent multi-query strategy that employs an extensive tree search algorithm. Intriguingly, our results suggest that instructing an LLM using an algorithm can lead to performance surpassing that of the algorithm itself, hinting at LLM's inherent ability to weave its intuition into optimized searches. We probe into the underpinnings of our method's efficacy and its nuances in application.

翻译：当前旨在超越“思维链”方法的文献，通常采用外部操作模式，通过暂停、修改并恢复生成过程来提升大型语言模型的推理能力。这种模式增加了查询请求次数，导致成本、内存和计算开销上升。针对这一问题，我们提出“思想算法”——一种通过算法推理路径驱动LLM的新型策略，开创了上下文学习的新模式。通过利用算法示例，我们挖掘LLM固有的递归动态特性，仅需一次或少量查询即可扩展其思维探索范围。我们的技术优于早期的单次查询方法，并与近期采用广泛树搜索算法的多次查询策略性能相当。有趣的是，实验结果表明，用算法指导LLM可使其性能超越算法本身，这暗示LLM具有将直觉融入优化搜索的固有能力。我们深入探究了该方法有效性的内在机制及其应用细节。