Large Language Models (LLMs) have emerged as promising assistants for scientific writing. However, there have been concerns regarding the quality and reliability of the generated text, one of which is the citation accuracy and faithfulness. While most recent work relies on methods such as LLM-as-a-Judge, the reliability of LLM-as-a-Judge alone is also in doubt. In this work, we reframe citation evaluation as a problem of citation attribution alignment, which assesses whether LLM-generated citations match those a human author would include for the same text. We propose CiteGuard, a retrieval-aware agent framework designed to provide more faithful grounding for citation validation. CiteGuard improves the prior baseline by 17%, and achieves up to 68.1% accuracy on the CiteME benchmark, approaching human-level performance (69.7%). It also enables the identification of alternative but valid citations and demonstrates generalization ability for cross-domain citation attribution.Our code is available at https://github.com/KathCYM/CiteGuard.
翻译:大型语言模型(LLMs)已成为科学写作领域极具潜力的辅助工具。然而,关于生成文本的质量与可靠性一直存在诸多关切,其中引用准确性与忠实度是核心问题之一。尽管近期研究多依赖LLM-as-a-Judge等方法,但单纯依靠此类方法的可靠性仍存疑。本研究将引用评估重新定义为引用归因对齐问题,旨在评估LLM生成的引用是否与人类作者针对相同文本会采用的引用相匹配。我们提出CiteGuard——一个具备检索感知能力的智能体框架,旨在为引用验证提供更可靠的依据。CiteGuard将现有基线性能提升了17%,在CiteME基准测试中达到68.1%的准确率,接近人类水平(69.7%)。该框架不仅能识别有效替代引用,还展现出跨领域引用归因的泛化能力。代码已开源:https://github.com/KathCYM/CiteGuard。