Recent advancements in Large Language Models (LLMs) have significantly enhanced their ability to perform complex reasoning tasks, transitioning from fast and intuitive thinking (System 1) to slow and deep reasoning (System 2). While System 2 reasoning improves task accuracy, it often incurs substantial computational costs due to its slow thinking nature and inefficient or unnecessary reasoning behaviors. In contrast, System 1 reasoning is computationally efficient but leads to suboptimal performance. Consequently, it is critical to balance the trade-off between performance (benefits) and computational costs (budgets), giving rise to the concept of reasoning economy. In this survey, we provide a comprehensive analysis of reasoning economy in both the post-training and test-time inference stages of LLMs, encompassing i) the cause of reasoning inefficiency, ii) behavior analysis of different reasoning patterns, and iii) potential solutions to achieve reasoning economy. By offering actionable insights and highlighting open challenges, we aim to shed light on strategies for improving the reasoning economy of LLMs, thereby serving as a valuable resource for advancing research in this evolving area. We also provide a public repository to continually track developments in this fast-evolving field.

翻译：近年来，大语言模型（LLMs）在执行复杂推理任务的能力上取得了显著进步，实现了从快速、直觉性思维（系统1）到缓慢、深度推理（系统2）的转变。虽然系统2推理提高了任务准确性，但由于其缓慢思考的特性以及低效或不必要的推理行为，往往会产生巨大的计算成本。相比之下，系统1推理计算效率高，但会导致性能欠佳。因此，平衡性能（收益）与计算成本（预算）之间的权衡至关重要，这催生了推理经济性的概念。在本综述中，我们对LLMs在后训练和测试时推理两个阶段的推理经济性进行了全面分析，涵盖：i) 推理低效的成因，ii) 不同推理模式的行为分析，以及iii) 实现推理经济性的潜在解决方案。通过提供可操作的见解并突出未解决的挑战，我们旨在阐明改进LLMs推理经济性的策略，从而为推动这一不断发展的领域的研究提供宝贵资源。我们还提供了一个公共知识库，以持续追踪这一快速演进领域的最新进展。