Retrieval-Augmented Generation (RAG) enhances Large Language Models (LLMs) with external knowledge but remains vulnerable to low-authority sources that can propagate misinformation. We investigate whether LLMs can perceive information authority - a capability extending beyond semantic understanding. To address this, we introduce AuthorityBench, a comprehensive benchmark for evaluating LLM authority perception comprising three datasets: DomainAuth (10K web domains with PageRank-based authority), EntityAuth (22K entities with popularity-based authority), and RAGAuth (120 queries with documents of varying authority for downstream evaluation). We evaluate five LLMs using three judging methods (PointJudge, PairJudge, ListJudge) across multiple output formats. Results show that ListJudge and PairJudge with PointScore output achieve the strongest correlation with ground-truth authority, while ListJudge offers optimal cost-effectiveness. Notably, incorporating webpage text consistently degrades judgment performance, suggesting authority is distinct from textual style. Downstream experiments on RAG demonstrate that authority-guided filtering largely improves answer accuracy, validating the practical importance of authority perception for reliable knowledge retrieval. Code and benchmark are available at: https://github.com/Trustworthy-Information-Access/AuthorityBench.

翻译：检索增强生成（RAG）通过引入外部知识提升了大型语言模型（LLM）的能力，但仍易受低权威性来源的影响，从而可能传播错误信息。我们探究LLM是否具备信息权威性感知能力——这一能力超越了语义理解层面。为此，我们提出了AuthorityBench，一个用于评估LLM权威性感知的综合基准，包含三个数据集：DomainAuth（基于PageRank权威性的10K网络域名）、EntityAuth（基于流行度权威性的22K实体）以及RAGAuth（包含120个查询及不同权威性文档的下游评估数据集）。我们采用三种评判方法（PointJudge、PairJudge、ListJudge）结合多种输出格式对五个LLM进行了评估。结果表明，ListJudge与PairJudge配合PointScore输出能与真实权威性实现最强相关性，且ListJudge在成本效益上最优。值得注意的是，引入网页文本会持续降低评判性能，这表明权威性与文本风格存在本质区别。下游RAG实验证实，基于权威性的过滤可显著提升答案准确率，验证了权威性感知对可靠知识检索的实践重要性。代码与基准数据集发布于：https://github.com/Trustworthy-Information-Access/AuthorityBench。