In biological tasks, data is rarely plentiful as it is generated from hard-to-gather measurements. Therefore, pre-training foundation models on large quantities of available data and then transfer to low-data downstream tasks is a promising direction. However, how to design effective foundation models for molecular learning remains an open question, with existing approaches typically focusing on models with large parameter capacities. In this work, we propose $\texttt{MiniMol}$, a foundational model for molecular learning with 10 million parameters. $\texttt{MiniMol}$ is pre-trained on a mix of roughly 3300 sparsely defined graph- and node-level tasks of both quantum and biological nature. The pre-training dataset includes approximately 6 million molecules and 500 million labels. To demonstrate the generalizability of $\texttt{MiniMol}$ across tasks, we evaluate it on downstream tasks from the Therapeutic Data Commons (TDC) ADMET group showing significant improvements over the prior state-of-the-art foundation model across 17 tasks. $\texttt{MiniMol}$ will be a public and open-sourced model for future research.

翻译：在生物任务中，数据通常因源自难以获取的测量而稀缺。因此，在大量可用数据上预训练基础模型，再迁移至低数据下游任务，是一个有前景的方向。然而，如何为分子学习设计有效的基础模型仍是一个开放问题，现有方法通常聚焦于具有大参数容量的模型。本研究提出$\texttt{MiniMol}$，一种包含1000万参数的分子学习基础模型。该模型在约3300个稀疏定义的图级和节点级任务（涵盖量子与生物性质）的组合数据上预训练。预训练数据集包含约600万个分子和5亿个标签。为展示$\texttt{MiniMol}$的跨任务泛化能力，我们在治疗性数据共同体（TDC）ADMET组的下游任务上评估该模型，结果表明其在17项任务中显著优于此前最先进的基础模型。$\texttt{MiniMol}$将作为公开开源模型供未来研究使用。