Recently, the remarkable capabilities of large language models (LLMs) have been illustrated across a variety of research domains such as natural language processing, computer vision, and molecular modeling. We extend this paradigm by utilizing LLMs for material property prediction by introducing our model Materials Informatics Transformer (MatInFormer). Specifically, we introduce a novel approach that involves learning the grammar of crystallography through the tokenization of pertinent space group information. We further illustrate the adaptability of MatInFormer by incorporating task-specific data pertaining to Metal-Organic Frameworks (MOFs). Through attention visualization, we uncover the key features that the model prioritizes during property prediction. The effectiveness of our proposed model is empirically validated across 14 distinct datasets, hereby underscoring its potential for high throughput screening through accurate material property prediction.

翻译：近期，大语言模型（LLMs）在自然语言处理、计算机视觉和分子建模等众多研究领域展现出卓越能力。我们通过引入材料信息学Transformer（MatInFormer）模型，将这一范式拓展至材料性能预测领域。具体而言，我们提出了一种创新方法，通过对相关空间群信息进行分词处理来学习晶体学语法。通过整合金属有机框架（MOFs）的任务特定数据，进一步展示了MatInFormer的适应性。通过注意力可视化，我们揭示了模型在性能预测中优先关注的关键特征。我们提出的模型有效性已在14个不同数据集上得到实证验证，凸显了其通过精准材料性能预测进行高通量筛选的潜力。