The decompose-then-verify strategy for verification of Large Language Model (LLM) generations decomposes claims that are then independently verified. Decontextualization augments text (claims) to ensure it can be verified outside of the original context, enabling reliable verification. While decomposition and decontextualization have been explored independently, their interactions in a complete system have not been investigated. Their conflicting purposes can create tensions: decomposition isolates atomic facts while decontextualization inserts relevant information. Furthermore, a decontextualized subclaim presents a challenge to the verification step: what part of the augmented text should be verified as it now contains multiple atomic facts? We conduct an evaluation of different decomposition, decontextualization, and verification strategies and find that the choice of strategy matters in the resulting factuality scores. Additionally, we introduce DnDScore, a decontextualization aware verification method which validates subclaims in the context of contextual information.
翻译:针对大型语言模型生成内容的事实性验证,分解后验证策略将生成内容中的主张分解为独立可验证的子主张。去语境化则通过增强文本(主张)内容,确保其脱离原始语境后仍能被可靠验证。尽管分解与去语境化方法已分别得到研究,但二者在完整系统中的交互机制尚未被深入探讨。这两种方法的目标冲突可能引发矛盾:分解旨在分离原子事实,而去语境化则需要插入相关信息。此外,经过去语境化处理的子主张对验证步骤提出了新挑战:增强后的文本包含多个原子事实时,应验证哪部分内容?本研究系统评估了不同分解策略、去语境化方法与验证机制的组合效果,发现策略选择对最终事实性评分具有显著影响。在此基础上,我们提出DnDScore——一种融合语境感知的验证方法,能够在语境信息框架下对子主张进行有效性验证。