While inference-time thinking allows Large Language Models (LLMs) to address complex problems, the extended thinking process can be unreliable or inconsistent because of the model's probabilistic nature, especially near its knowledge boundaries. Existing approaches attempt to mitigate this by having the model critique its own reasoning to make corrections. However, such self-critique inherits the same biases of the original output, known as the introspection illusion. Moving beyond such introspection and inspired by core methodologies in ethology, we propose an externalist three-step framework Distillation-Reinforcement-Reasoning (DRR). Rather than relying on a model's introspection, DRR evaluates its observable behaviors to provide corrective feedback. DRR first distills the reasoner's behavioral traces, then trains a lightweight, external Discriminative Model (DM). At inference time, this DM acts as a critic, identifying and rejecting suspicious reasoning steps. This external feedback compels the LLM to discard flawed pathways and explore alternatives, thereby enhancing reasoning quality without altering the base model. Experiments on multiple reasoning benchmarks show that our framework significantly outperforms prominent self-critique methods. Benefiting from a lightweight and annotation-free design, DRR offers a scalable and adaptable solution for improving the reliability of reasoning in a wide range of LLMs.

翻译：尽管推理时思考使大型语言模型（LLMs）能够处理复杂问题，但由于模型固有的概率特性，尤其是在其知识边界附近，扩展的思考过程可能不可靠或不一致。现有方法试图通过让模型批判自身推理以进行修正来缓解这一问题。然而，这种自我批判继承了原始输出的相同偏见，即所谓的内省错觉。受行为学核心方法的启发，我们提出了一种超越内省的外部主义三步框架——蒸馏-强化-推理（DRR）。DRR不依赖模型的内省，而是通过评估其可观察行为来提供纠正性反馈。该框架首先蒸馏推理器的行为轨迹，然后训练一个轻量级的外部判别模型（DM）。在推理时，DM充当批评者，识别并拒绝可疑的推理步骤。这种外部反馈迫使LLM放弃有缺陷的推理路径并探索替代方案，从而在不改变基础模型的情况下提升推理质量。在多个推理基准上的实验表明，我们的框架显著优于主流的自我批判方法。得益于轻量级且无需标注的设计，DRR为广泛提升各类LLMs的推理可靠性提供了一种可扩展且适应性强的解决方案。