Entity-aware document retrieval uses query-associated entities as ranking signals, assuming that semantically relevant entities are also useful retrieval signals. We show this assumption is insufficient- and explain why. Unlike terms, which are ground-truth observations, entity links are hypotheses produced by an imperfect linker: an entity can be topically central yet provide no discriminative signal if the linker fires indiscriminately across relevant and non-relevant documents. We formalize this as a distinction between Conceptual Entity Relevance (CER)- whether an entity is topically related to a query- and Observable Entity Relevance (OER)- whether its observed presence in a collection discriminates relevant from non-relevant documents. Across four collections and annotation sources including human entity judgments, CER and OER exhibit near-chance agreement ($κ\approx 0$), while OER operationalizations agree substantially ($κ\approx 0.5$), confirming CER as the systematic outlier. CER-based supervision selects topically plausible but weakly discriminative entities, pruning fewer than 4% of non-relevant documents on some collections. Aligning supervision with OER improves non-relevant pruning by up to 10x and open-world MAP by 0.051 over BM25. Our findings motivate a shift from conceptual to observable notions of entity relevance in entity-aware retrieval.

翻译：实体感知文档检索将查询关联实体作为排序信号，假设语义相关的实体在检索中同样具有判别效力。本文证明该假设存在根本性缺陷并阐明其成因。不同于作为真实观测的词项，实体链接是经由不完全链接器生成的假设性输出：当链接器在相关与非相关文档间无差异触发时，即使实体在主题层面具有核心地位，也无法提供有效判别信号。我们通过概念实体相关性（CER）与可观测实体相关性（OER）的二分法形式化该问题——前者衡量实体与查询的主题关联度，后者衡量实体在文档集合中的实际出现是否能区分相关与非相关文档。在包含人工实体标注的四个数据集与多种标注源中，CER与OER呈现近乎随机的吻合度（κ≈0），而OER的不同操作化方案间高度一致（κ≈0.5），验证CER是系统性偏差源。基于CER的监督策略会筛选出主题合理但判别力薄弱的实体，在部分集合中仅能过滤不足4%的非相关文档。将监督信号对齐OER后，非相关文档过滤率提升至10倍，开放世界MAP较BM25基线提升0.051。本研究表明实体感知检索应从概念相关性范式转向可观测相关性范式。