Evaluating the factuality of long-form output generated by large language models (LLMs) remains challenging, particularly when responses are open-ended and contain many fine-grained factual statements. Existing evaluation methods primarily focus on precision: they decompose a response into atomic claims and verify each claim against external knowledge sources such as Wikipedia. However, this overlooks an equally important dimension of factuality: recall, whether the generated response covers the relevant facts that should be included. We propose a comprehensive factuality evaluation framework that jointly measures precision and recall. Our method leverages external knowledge sources to construct reference facts and determine whether they are captured in generated text. We further introduce an importance-aware weighting scheme based on relevance and salience. Our analysis reveals that current LLMs perform substantially better on precision than on recall, suggesting that factual incompleteness remains a major limitation of long-form generation and that models are generally better at covering highly important facts than the full set of relevant facts.

翻译：评估大语言模型（LLMs）生成的长文本输出的事实性仍然具有挑战性，尤其是当回答是开放式的且包含大量细粒度事实陈述时。现有的评估方法主要关注精确度：它们将回答分解为原子化声明，并对照外部知识源（如维基百科）逐一验证每个声明。然而，这忽略了事实性的另一个同等重要的维度：召回率，即生成的回答是否涵盖了应当包含的相关事实。我们提出了一种综合事实性评估框架，可同时衡量精确度和召回率。我们的方法利用外部知识源构建参考事实，并判断这些事实是否被生成的文本所涵盖。我们进一步引入了一种基于相关性和显著性的重要性感知加权方案。我们的分析表明，当前大语言模型在精确度上的表现远优于召回率，这表明事实不完整性仍是长文本生成的主要缺陷，且模型通常更擅长覆盖高重要性事实，而非完整的相关事实集合。