Large molecular representation models pre-trained on massive unlabeled data have shown great success in predicting molecular properties. However, these models may tend to overfit the fine-tuning data, resulting in over-confident predictions on test data that fall outside of the training distribution. To address this issue, uncertainty quantification (UQ) methods can be used to improve the models' calibration of predictions. Although many UQ approaches exist, not all of them lead to improved performance. While some studies have included UQ to improve molecular pre-trained models, the process of selecting suitable backbone and UQ methods for reliable molecular uncertainty estimation remains underexplored. To address this gap, we present MUBen, which evaluates different UQ methods for state-of-the-art backbone molecular representation models to investigate their capabilities. By fine-tuning various backbones using different molecular descriptors as inputs with UQ methods from different categories, we critically assess the influence of architectural decisions and training strategies. Our study offers insights for selecting UQ for backbone models, which can facilitate research on uncertainty-critical applications in fields such as materials science and drug discovery.

翻译：在大规模无标注数据上预训练的大型分子表示模型在预测分子性质方面取得了显著成功。然而，这些模型可能倾向于过拟合微调数据，导致对超出训练分布的测试数据做出过于自信的预测。为解决该问题，可借助不确定性量化（UQ）方法来改善模型的预测校准能力。尽管存在多种UQ方法，但并非所有方法都能提升性能。虽然已有研究将UQ用于改进分子预训练模型，但如何选择可靠的分子不确定性估计所需的骨干模型与UQ方法仍待探索。为填补这一空白，我们提出MUBen，该系统评估不同UQ方法在先进骨干分子表示模型上的性能。通过以不同分子描述符作为输入，结合不同类别的UQ方法微调多种骨干模型，我们深入分析了架构决策与训练策略的影响。本研究为骨干模型选择UQ方法提供了见解，可促进材料科学与药物发现等对不确定性敏感领域的研究。