Factual hallucination remains a central challenge for large language models (LLMs). Existing mitigation approaches primarily rely on either external post-hoc verification or mapping uncertainty directly to abstention during fine-tuning, often resulting in overly conservative behavior. We propose VeriFY, a training-time framework that teaches LLMs to reason about factual uncertainty through consistency-based self-verification. VeriFY augments training with structured verification traces that guide the model to produce an initial answer, generate and answer a probing verification query, issue a consistency judgment, and then decide whether to answer or abstain. To address the risk of reinforcing hallucinated content when training on augmented traces, we introduce a stage-level loss masking approach that excludes hallucinated answer stages from the training objective while preserving supervision over verification behavior. Across multiple model families and scales, VeriFY reduces factual hallucination rates by 9.7 to 53.3 percent, with only modest reductions in recall (0.4 to 5.7 percent), and generalizes across datasets when trained on a single source. The source code, training data, and trained model checkpoints will be released upon acceptance.

翻译：事实性幻觉仍然是大型语言模型面临的核心挑战。现有的缓解方法主要依赖于外部事后验证或在微调期间直接将不确定性映射为弃答，这通常会导致模型行为过于保守。我们提出了VeriFY，一种训练时框架，通过基于一致性的自我验证来教导LLM推理事实不确定性。VeriFY通过结构化的验证轨迹来增强训练，这些轨迹引导模型产生初始答案、生成并回答一个探测性验证查询、发布一致性判断，然后决定是回答还是弃答。为了解决在增强轨迹上训练时可能强化幻觉内容的风险，我们引入了一种阶段级损失掩蔽方法，该方法将幻觉答案阶段从训练目标中排除，同时保留对验证行为的监督。在多个模型家族和规模上，VeriFY将事实性幻觉率降低了9.7%至53.3%，而召回率仅适度下降（0.4%至5.7%），并且在单一数据源上训练后能跨数据集泛化。源代码、训练数据和训练好的模型检查点将在论文被接受后发布。