Academic peer review remains the cornerstone of scholarly validation, yet the field faces some challenges in data and methods. From the data perspective, existing research is hindered by the scarcity of large-scale, verified benchmarks and oversimplified evaluation metrics that fail to reflect real-world editorial workflows. To bridge this gap, we present OmniReview, a comprehensive dataset constructed by integrating multi-source academic platforms encompassing comprehensive scholarly profiles through the disambiguation pipeline, yielding 202, 756 verified review records. Based on this data, we introduce a three-tier hierarchical evaluaion framework to assess recommendations from recall to precise expert identification. From the method perspective, existing embedding-based approaches suffer from the information bottleneck of semantic compression and limited interpretability. To resolve these method limitations, we propose Profiling Scholars with Multi-gate Mixture-of-Experts (Pro-MMoE), a novel framework that synergizes Large Language Models (LLMs) with Multi-task Learning. Specifically, it utilizes LLM-generated semantic profiles to preserve fine-grained expertise nuances and interpretability, while employing a Task-Adaptive MMoE architecture to dynamically balance conflicting evaluation goals. Comprehensive experiments demonstrate that Pro-MMoE achieves state-of-the-art performance across six of seven metrics, establishing a new benchmark for realistic reviewer recommendation.
翻译:学术同行评审作为学术验证的基石,其领域在数据与方法层面仍面临若干挑战。从数据视角看,现有研究受限于大规模、已验证基准的稀缺性,以及过度简化的评估指标难以反映现实编辑工作流程。为弥合此差距,我们提出了OmniReview——一个通过整合多源学术平台、经由消歧管道构建涵盖完整学者画像的综合数据集,包含202,756条已验证审稿记录。基于此数据,我们引入了一个三层级评估框架,用于从召回率到精确专家识别的多维度推荐评估。从方法视角看,现有基于嵌入的方法受限于语义压缩的信息瓶颈与有限的解释性。为解决这些方法局限,我们提出了基于多门混合专家系统的学者画像建模框架(Pro-MMoE),该新颖框架将大语言模型(LLMs)与多任务学习相协同。具体而言,它利用LLM生成的语义画像保留细粒度专业领域差异与可解释性,同时采用任务自适应MMoE架构动态平衡相互冲突的评估目标。综合实验表明,Pro-MMoE在七项评估指标中的六项达到最优性能,为现实审稿人推荐确立了新的基准。