Large language models (LLMs) are widely used in knowledge-intensive applications but often generate factually incorrect responses. A promising approach to rectify these flaws is correcting LLMs using feedback. Therefore, in this paper, we introduce FactCorrector, a new post-hoc correction method that adapts across domains without retraining and leverages structured feedback about the factuality of the original response to generate a correction. To support rigorous evaluations of factuality correction methods, we also develop the VELI5 benchmark, a novel dataset containing systematically injected factual errors and ground-truth corrections. Experiments on VELI5 and several popular long-form factuality datasets show that the FactCorrector approach significantly improves factual precision while preserving relevance, outperforming strong baselines. We release our code at https://ibm.biz/factcorrector.
翻译:大语言模型(LLMs)在知识密集型应用中已被广泛采用,但其生成的响应常包含事实性错误。一种有前景的修正途径是利用反馈对LLMs进行校正。为此,本文提出FactCorrector——一种新颖的事后校正方法,该方法无需重新训练即可跨领域适应,并利用关于原始响应事实性的结构化反馈来生成修正。为支持对事实性校正方法的严格评估,我们还构建了VELI5基准数据集,这是一个包含系统性注入的事实错误与真实校正的新颖数据集。在VELI5及多个主流长文本事实性数据集上的实验表明,FactCorrector方法在保持相关性的同时显著提升了事实精确度,其性能优于现有强基线模型。代码发布于 https://ibm.biz/factcorrector。