Molecular Property Prediction (MPP) task involves predicting biochemical properties based on molecular features, such as molecular graph structures, contributing to the discovery of lead compounds in drug development. To address data scarcity and imbalance in MPP, some studies have adopted Graph Neural Networks (GNN) as an encoder to extract commonalities from molecular graphs. However, these approaches often use a separate predictor for each task, neglecting the shared characteristics among predictors corresponding to different tasks. In response to this limitation, we introduce the GNN-MoCE architecture. It employs the Mixture of Collaborative Experts (MoCE) as predictors, exploiting task commonalities while confronting the homogeneity issue in the expert pool and the decision dominance dilemma within the expert group. To enhance expert diversity for collaboration among all experts, the Expert-Specific Projection method is proposed to assign a unique projection perspective to each expert. To balance decision-making influence for collaboration within the expert group, the Expert-Specific Loss is presented to integrate individual expert loss into the weighted decision loss of the group for more equitable training. Benefiting from the enhancements of MoCE in expert creation, dynamic expert group formation, and experts' collaboration, our model demonstrates superior performance over traditional methods on 24 MPP datasets, especially in tasks with limited data or high imbalance.
翻译:分子性质预测(MPP)任务旨在基于分子特征(如分子图结构)预测生化性质,有助于药物开发中先导化合物的发现。针对MPP中数据稀缺和不平衡问题,部分研究采用图神经网络(GNN)作为编码器提取分子图的共性特征。然而,这些方法常为每个任务使用独立的预测器,忽视了不同任务对应预测器之间的共享特性。为解决这一局限,我们提出GNN-MoCE架构。该架构采用协作专家混合模型(MoCE)作为预测器,在利用任务共性的同时,应对专家池中的同质性问题以及专家群体内的决策主导困境。为增强专家多样性以促进所有专家间的协作,我们提出专家专用投影方法,为每位专家分配独特的投影视角。为平衡专家群体内协作决策的影响力,我们提出专家专用损失函数,将单个专家损失整合到群体的加权决策损失中,以实现更公平的训练。得益于MoCE在专家创建、动态专家群体形成及专家协作方面的改进,我们的模型在24个MPP数据集上展现出优于传统方法的性能,尤其在数据量有限或高度不平衡的任务中表现突出。