Protein language models have demonstrated significant potential in the field of protein engineering. However, current protein language models primarily operate at the residue scale, which limits their ability to provide information at the atom level. This limitation prevents us from fully exploiting the capabilities of protein language models for applications involving both proteins and small molecules. In this paper, we propose ESM-AA (ESM All-Atom), a novel approach that enables atom-scale and residue-scale unified molecular modeling. ESM-AA achieves this by pre-training on multi-scale code-switch protein sequences and utilizing a multi-scale position encoding to capture relationships among residues and atoms. Experimental results indicate that ESM-AA surpasses previous methods in protein-molecule tasks, demonstrating the full utilization of protein language models. Further investigations reveal that through unified molecular modeling, ESM-AA not only gains molecular knowledge but also retains its understanding of proteins.

翻译：蛋白质语言模型在蛋白质工程领域展现了巨大潜力。然而，当前蛋白质语言模型主要作用于残基层面，这限制了其在原子层级提供信息的能力。这一局限阻碍了我们充分利用蛋白质语言模型处理涉及蛋白质与小分子协同应用的潜力。本文提出ESM-AA（ESM全原子模型），一种实现原子尺度与残基尺度统一分子建模的新方法。ESM-AA通过对多尺度代码切换蛋白质序列进行预训练，并采用多尺度位置编码捕获残基与原子间关系来实现这一目标。实验结果表明，ESM-AA在蛋白质-分子任务上超越以往方法，充分展现了蛋白质语言模型的利用价值。进一步研究表明，通过统一分子建模，ESM-AA不仅习得了分子知识，还保留了对蛋白质的理解能力。