Automation is one of the cornerstones of contemporary material discovery. Bayesian optimization (BO) is an essential part of such workflows, enabling scientists to leverage prior domain knowledge into efficient exploration of a large molecular space. While such prior knowledge can take many forms, there has been significant fanfare around the ancillary scientific knowledge encapsulated in large language models (LLMs). However, existing work thus far has only explored LLMs for heuristic materials searches. Indeed, recent work obtains the uncertainty estimate -- an integral part of BO -- from point-estimated, non-Bayesian LLMs. In this work, we study the question of whether LLMs are actually useful to accelerate principled Bayesian optimization in the molecular space. We take a sober, dispassionate stance in answering this question. This is done by carefully (i) viewing LLMs as fixed feature extractors for standard but principled BO surrogate models and by (ii) leveraging parameter-efficient finetuning methods and Bayesian neural networks to obtain the posterior of the LLM surrogate. Our extensive experiments with real-world chemistry problems show that LLMs can be useful for BO over molecules, but only if they have been pretrained or finetuned with domain-specific data.

翻译：自动化是当代材料发现的基石之一。贝叶斯优化（BO）是此类工作流程的重要组成部分，使科学家能够利用先验领域知识高效探索庞大的分子空间。尽管这类先验知识有多种表现形式，但大型语言模型（LLM）所蕴含的辅助科学知识已引起广泛关注。然而，现有研究仅探索了将LLM用于启发式材料搜索。实际上，近期研究从点估计的非贝叶斯LLM中获取了不确定性估计（BO的关键组成部分）。本研究旨在探讨LLM是否真正有助于加速分子空间中的原则性贝叶斯优化。我们以冷静、客观的态度回答这一问题，通过以下方式实现：（i）将LLM视为标准但原则性BO代理模型的固定特征提取器；（ii）利用参数高效微调方法和贝叶斯神经网络获取LLM代理的后验分布。基于真实化学问题的广泛实验表明：LLM对分子空间中的BO具有实用价值，但前提是它们必须经过领域特定数据的预训练或微调。