Large Language Models (LLMs) have shown impressive reasoning capabilities, yet existing prompting methods face a critical trade-off: simple approaches often struggle with complex tasks and reasoning stability, while more sophisticated methods require multiple inferences and substantial computational resources, limiting their practical deployment. To address this challenge, we propose Derailer-Rerailer, a novel framework that adaptively balances reasoning accuracy and computational efficiency. At its core, our framework employs a lightweight Derailer mechanism to assess reasoning stability and selectively triggers an advanced Rerailer verification process only when necessary, thereby optimizing computational resource usage. Extensive evaluation across both open and closed-source models on more than 20 categories of mathematical, symbolic, and commonsense reasoning tasks demonstrates our framework's effectiveness: Derailer-Rerailer achieves significant accuracy improvements (8-11\% across various reasoning tasks) while maintaining 2-3 times better efficiency than existing verification methods, with particularly strong performance in mathematical and symbolic reasoning, offering a practical solution for enhancing LLM reasoning reliability while significantly reducing computational overhead.
翻译:大型语言模型(LLMs)已展现出卓越的推理能力,但现有提示方法面临一个关键权衡:简单方法常难以处理复杂任务并保证推理稳定性,而更复杂的方法需要多次推理和大量计算资源,限制了实际部署。为应对这一挑战,我们提出脱轨器-复轨器(Derailer-Rerailer)——一种自适应平衡推理准确性与计算效率的新型框架。该框架的核心采用轻量级脱轨器机制评估推理稳定性,并仅在必要时触发高级复轨器验证过程,从而优化计算资源使用。通过对开源和闭源模型在超过20类数学、符号与常识推理任务上的广泛评估,本框架展现出显著效果:脱轨器-复轨器在各类推理任务中实现8-11%的准确率提升,同时保持比现有验证方法高2-3倍的效率,在数学与符号推理任务中表现尤为突出,为增强LLM推理可靠性并显著降低计算开销提供了实用解决方案。