Histopathology plays a central role in clinical medicine and biomedical research. While artificial intelligence shows promising results on many pathological tasks, generalization and dealing with rare diseases, where training data is scarce, remains a challenge. Distilling knowledge from unlabelled data into a foundation model before learning from, potentially limited, labelled data provides a viable path to address these challenges. In this work, we extend the state of the art of foundation models for digital pathology whole slide images by semi-automated data curation and incorporating pathologist domain knowledge. Specifically, we combine computational and pathologist domain knowledge (1) to curate a diverse dataset of 133k slides corresponding to 1.2 billion image patches covering data from different fixation, staining, and scanning protocols as well as data from different indications and labs across the EU and US, (2) for grouping semantically similar slides and tissue patches, and (3) to augment the input images during training. We evaluate the resulting model on a set of public and internal benchmarks and show that although our foundation model is trained with an order of magnitude less slides, it performs on par or better than competing models. We expect that scaling our approach to more data and larger models will further increase its performance and capacity to deal with increasingly complex real world tasks in diagnostics and biomedical research.

翻译：组织病理学在临床医学和生物医学研究中起着核心作用。尽管人工智能在众多病理学任务上展现出令人鼓舞的成果，但泛化能力以及在训练数据稀缺的罕见疾病处理方面仍面临挑战。在利用可能有限的标注数据进行学习之前，先从无标注数据中提炼知识并构建基础模型，是应对这些挑战的一条可行路径。在本研究中，我们通过半自动化数据整理并融入病理学家的领域知识，拓展了用于数字病理学全切片图像的基础模型的最新技术水平。具体而言，我们结合计算与病理学家的领域知识：（1）整理了涵盖133,000张切片的多样化数据集（对应12亿图像块），这些数据包含不同固定、染色和扫描方案，以及来自欧盟和美国不同实验室与适应症的数据；（2）对语义相似的切片和组织块进行分组；（3）在训练期间增强输入图像。我们在公开及内部基准测试上评估了所得到的模型，结果表明，尽管我们的基础模型所用切片数量少一个数量级，但其性能与竞争模型相当甚至更优。我们预期，将本方法扩展到更多数据和更大规模的模型，将进一步提升其性能及应对诊断与生物医学研究中日益复杂的实际任务的能力。