Adsorption energy, a reactivity descriptor, should be accurately assessed for efficient catalyst screening. This evaluation requires determining the lowest energy across various adsorption configurations on the catalytic surface. While graph neural networks (GNNs) have gained popularity as a machine learning approach for computing the energy of catalyst systems, they rely heavily on atomic spatial coordinates and often lack clarity in their interpretations. Recent advancements in language models have broadened their applicability to predicting catalytic properties, allowing us to bypass the complexities of graph representation. These models are adept at handling textual data, making it possible to incorporate observable features in a human-readable format. However, language models encounter challenges in accurately predicting the energy of adsorption configurations, typically showing a high mean absolute error (MAE) of about 0.71 eV. Our study addresses this limitation by introducing a self-supervised multi-modal learning approach, termed graph-assisted pretraining. This method significantly reduces the MAE to 0.35 eV through a combination of data augmentation, achieving comparable accuracy with DimeNet++ while using 0.4% of its training data size. Furthermore, the Transformer encoder at the core of the language model can provide insights into the feature focus through its attention scores. This analysis shows that our multimodal training effectively redirects the model's attention toward relevant adsorption configurations from adsorbate-related features, enhancing prediction accuracy and interpretability.

翻译：吸附能作为反应活性的描述符，需被精确评估以实现高效催化剂筛选。这一评估要求确定催化表面上各种吸附构型中的最低能量。尽管图神经网络（GNNs）作为计算催化剂系统能量的主流机器学习方法广受欢迎，但其高度依赖原子空间坐标且常缺乏清晰的解释性。语言模型的最新进展已扩展至催化性能预测领域，使我们能够绕过图表示的复杂性。这类模型擅长处理文本数据，从而能够以人类可读格式整合可观测特征。然而，语言模型在精确预测吸附构型能量方面仍面临挑战，其平均绝对误差（MAE）通常高达约0.71 eV。本研究通过引入一种名为图辅助预训练的自监督多模态学习方法突破了这一限制。该方法通过数据增强将MAE显著降至0.35 eV，在仅使用DimeNet++训练数据量0.4%的情况下达到与其相当的精度。此外，作为语言模型核心的Transformer编码器可通过注意力得分揭示特征聚焦方向。分析表明，我们的多模态训练有效将模型注意力从吸附质相关特征转向相关吸附构型，从而提升了预测精度与可解释性。