Recent advancements in biological research leverage the integration of molecules, proteins, and natural language to enhance drug discovery. However, current models exhibit several limitations, such as the generation of invalid molecular SMILES, underutilization of contextual information, and equal treatment of structured and unstructured knowledge. To address these issues, we propose $\mathbf{BioT5}$, a comprehensive pre-training framework that enriches cross-modal integration in biology with chemical knowledge and natural language associations. $\mathbf{BioT5}$ utilizes SELFIES for $100%$ robust molecular representations and extracts knowledge from the surrounding context of bio-entities in unstructured biological literature. Furthermore, $\mathbf{BioT5}$ distinguishes between structured and unstructured knowledge, leading to more effective utilization of information. After fine-tuning, BioT5 shows superior performance across a wide range of tasks, demonstrating its strong capability of capturing underlying relations and properties of bio-entities. Our code is available at $\href{https://github.com/QizhiPei/BioT5}{Github}$.

翻译：近期生物学研究的前沿进展，通过整合分子、蛋白质与自然语言以增强药物发现。然而，现有模型存在若干局限性，例如生成无效分子SMILES、上下文信息利用不足，以及结构化与非结构化知识的等同处理。为解决上述问题，我们提出$\mathbf{BioT5}$——一个通过化学知识与自然语言关联来丰富生物学跨模态整合的综合预训练框架。$\mathbf{BioT5}$采用SELFIES实现$100\%$鲁棒的分子表征，并从非结构化生物文献中提取生物实体周边上下文的知识。此外，$\mathbf{BioT5}$区分结构化与非结构化知识，从而实现更高效的信息利用。经微调后，BioT5在广泛任务中展现出优越性能，证明其捕捉生物实体潜在关联与属性的强大能力。我们的代码已开源至$\href{https://github.com/QizhiPei/BioT5}{Github}$。