Foundation models (FMs) are able to leverage large volumes of unlabeled data to demonstrate superior performance across a wide range of tasks. However, FMs developed for biomedical domains have largely remained unimodal, i.e., independently trained and used for tasks on protein sequences alone, small molecule structures alone, or clinical data alone. To overcome this limitation of biomedical FMs, we present BioBridge, a novel parameter-efficient learning framework, to bridge independently trained unimodal FMs to establish multimodal behavior. BioBridge achieves it by utilizing Knowledge Graphs (KG) to learn transformations between one unimodal FM and another without fine-tuning any underlying unimodal FMs. Our empirical results demonstrate that BioBridge can beat the best baseline KG embedding methods (on average by around 76.3%) in cross-modal retrieval tasks. We also identify BioBridge demonstrates out-of-domain generalization ability by extrapolating to unseen modalities or relations. Additionally, we also show that BioBridge presents itself as a general purpose retriever that can aid biomedical multimodal question answering as well as enhance the guided generation of novel drugs.

翻译：基础模型（FMs）能够利用大量无标注数据，在各类任务中展现出卓越性能。然而，针对生物医学领域开发的基础模型大多仍局限于单模态——即分别独立训练并应用于蛋白质序列、小分子结构或临床数据等单一任务。为解决生物医学基础模型这一局限，我们提出BioBridge——一种高效的参数化学习框架，通过桥接独立训练的单模态基础模型来建立多模态行为。该框架利用知识图谱（KG）学习单模态基础模型之间的转换关系，而无需对底层单模态基础模型进行微调。实验结果表明，在跨模态检索任务中，BioBridge平均性能比最佳基线知识图谱嵌入方法高出约76.3%。我们还发现BioBridge可通过外推至未见模态或关系展现出跨领域泛化能力。此外，BioBridge作为通用检索工具，既能辅助生物医学多模态问答系统，也可增强新型药物的引导式生成。