Large reasoning models (LRMs) show strong capabilities in complex reasoning, yet their marginal gains on evidence-dependent factual questions are limited. We find this limitation is partially attributable to a reasoning-answer hit gap, where the model identifies the correct facts during reasoning but fails to incorporate them into the final response, thereby reducing factual fidelity. To address this issue, we propose MR-ALIGN, a Meta-Reasoning informed alignment framework that enhances factuality without relying on external verifiers. MR-ALIGN quantifies state transition probabilities along the model's thinking process and constructs a transition-aware implicit reward that reinforces beneficial reasoning patterns while suppressing defective ones at the atomic thinking segments. This re-weighting reshapes token-level signals into probability-aware segment scores, encouraging coherent reasoning trajectories that are more conducive to factual correctness. Empirical evaluations across four factual QA datasets and one long-form factuality benchmark show that MR-ALIGN consistently improves accuracy and truthfulness while reducing misleading reasoning. These results highlight that aligning the reasoning process itself, rather than merely the outputs, is pivotal for advancing factuality in LRMs.

翻译：大型推理模型（LRMs）在复杂推理任务中展现出强大能力，但其在依赖证据的事实性问题上的边际收益有限。我们发现这一局限部分归因于推理-答案命中鸿沟：模型在推理过程中识别出正确事实，却未能将其整合至最终回答中，从而降低了事实保真度。为解决该问题，我们提出MR-ALIGN——一种基于元推理的对齐框架，可在不依赖外部验证器的情况下增强事实性。MR-ALIGN通过量化模型思维过程中的状态转移概率，构建具有转移感知的隐式奖励机制，该机制在原子思维片段层面强化有益的推理模式，同时抑制缺陷模式。这种重加权策略将词元级信号重塑为概率感知的片段评分，从而鼓励生成更连贯且更有利于事实正确性的推理轨迹。在四个事实性问答数据集和一个长文本事实性基准上的实证评估表明，MR-ALIGN能持续提升准确率与真实性，同时减少误导性推理。这些结果凸显了对推理过程本身（而非仅对输出结果）进行对齐，对于提升LRMs的事实性具有关键意义。