Large Language Models (LLMs) have emerged as powerful assistants for scientific writing. However, concerns remain about the quality and reliability of the generated text, including citation accuracy and faithfulness. While most recent work relies on methods such as LLM-as-a-Judge, the reliability of LLM-as-a-Judge alone is also in doubt. In this work, we reframe citation evaluation as a problem of citation attribution alignment, which assesses whether LLM-generated citations match those a human author would include for the same text. We propose CiteGuard, a retrieval-aware agent framework designed to provide more faithful grounding for citation validation. CiteGuard improves over the prior baseline by 10 percentage points and achieves up to 68.1% accuracy on the CiteME benchmark, approaching human performance (69.2%). It also identifies alternative valid citations and demonstrates generalization ability for cross-domain citation attribution. Our code is available at https://github.com/KathCYM/CiteGuard.
翻译:大规模语言模型(LLMs)已成为科学写作的强大助手。然而,生成文本的质量与可靠性仍存隐忧,包括引用的准确性和忠实性。尽管近期研究多依赖"以LLM为裁判"等方法,但仅靠LLM评判的可靠性本身也受到质疑。本文将引用评估重新定义为引用归因对齐问题——即评估LLM生成的引用是否与人类作者针对相同文本可能采用的引用相匹配。我们提出CiteGuard,一种检索感知型智能体框架,旨在为引用验证提供更可靠的依据。CiteGuard在CiteME基准测试中相较于先前基线方法提升10个百分点,准确率达68.1%,接近人类表现(69.2%)。该框架还能识别替代有效引用,并展现出跨领域引用归因的泛化能力。我们的代码开源在https://github.com/KathCYM/CiteGuard。