Activity and property prediction models are the central workhorses in drug discovery and materials sciences, but currently they have to be trained or fine-tuned for new tasks. Without training or fine-tuning, scientific language models could be used for such low-data tasks through their announced zero- and few-shot capabilities. However, their predictive quality at activity prediction is lacking. In this work, we envision a novel type of activity prediction model that is able to adapt to new prediction tasks at inference time, via understanding textual information describing the task. To this end, we propose a new architecture with separate modules for chemical and natural language inputs, and a contrastive pre-training objective on data from large biochemical databases. In extensive experiments, we show that our method CLAMP yields improved predictive performance on few-shot learning benchmarks and zero-shot problems in drug discovery. We attribute the advances of our method to the modularized architecture and to our pre-training objective.

翻译：活性与性质预测模型是药物发现和材料科学中的核心工具，但目前模型需针对新任务进行训练或微调。科学语言模型凭借其宣称的零样本和小样本能力，可在无需训练或微调的情况下适用于此类低数据任务，但其在活性预测方面的预测质量仍存在不足。本文构想了一种新型活性预测模型，该模型能通过理解描述任务的文本信息，在推理阶段自适应地适应新预测任务。为此，我们提出了一种新架构，分别包含处理化学信息和自然语言输入的独立模块，并基于大规模生物化学数据库数据设计了一种对比预训练目标。通过大量实验证明，我们的方法CLAMP在药物发现的小样本学习基准和零样本问题中均取得了更优的预测性能。我们将该方法的优势归因于模块化架构和所提出的预训练目标。