Semantic relevance judgment for search is particularly challenging in knowledge-intensive scenarios, where accurate ranking requires not only semantic matching but also background grounding, multi-step reasoning, and well-calibrated decision boundaries. Existing relevance models mainly rely on direct label supervision or shallow semantic similarity, which limits their ability to handle implicit intent, factual equivalence, and fine-grained relevance distinctions. To address this issue, we propose \textsc{RAG-Match}, a three-stage framework that integrates knowledge-augmented pretraining, hierarchical reasoning alignment, and preference-based decision calibration for relevance modeling. The key idea is to first strengthen query-centered semantic grounding, then align the model with structured relevance reasoning, and finally correct decision-level inconsistencies in difficult boundary cases. Experimental results on a real-world search relevance benchmark show that \textsc{RAG-Match} consistently outperforms strong LLM-based baselines across multiple ranking metrics, demonstrating the effectiveness of combining knowledge injection, reasoning supervision, and preference optimization for fine-grained relevance judgment.
翻译:语义相关性判断在知识密集型搜索场景中尤为困难,准确排序不仅需要语义匹配,还需背景知识支撑、多步推理以及校准的决策边界。现有相关性模型主要依赖直接标签监督或浅层语义相似度,难以处理隐式意图、事实等价关系及细粒度相关性差异。针对该问题,我们提出\textsc{RAG-Match}三阶段框架,融合知识增强预训练、层次化推理对齐与基于偏好的决策校准进行相关性建模。其核心思想是:首先强化以查询为中心的语义背景理解,然后使模型与结构化相关性推理对齐,最后修正困难边界案例中的决策层不一致性。基于真实搜索相关性基准的实验结果表明,\textsc{RAG-Match}在多个排序指标上持续优于强LLM基线模型,验证了知识注入、推理监督与偏好优化相结合对细粒度相关性判断的有效性。