The accelerated adoption of digital pathology and advances in deep learning have enabled the development of powerful models for various pathology tasks across a diverse array of diseases and patient cohorts. However, model training is often difficult due to label scarcity in the medical domain and the model's usage is limited by the specific task and disease for which it is trained. Additionally, most models in histopathology leverage only image data, a stark contrast to how humans teach each other and reason about histopathologic entities. We introduce CONtrastive learning from Captions for Histopathology (CONCH), a visual-language foundation model developed using diverse sources of histopathology images, biomedical text, and notably over 1.17 million image-caption pairs via task-agnostic pretraining. Evaluated on a suite of 13 diverse benchmarks, CONCH can be transferred to a wide range of downstream tasks involving either or both histopathology images and text, achieving state-of-the-art performance on histology image classification, segmentation, captioning, text-to-image and image-to-text retrieval. CONCH represents a substantial leap over concurrent visual-language pretrained systems for histopathology, with the potential to directly facilitate a wide array of machine learning-based workflows requiring minimal or no further supervised fine-tuning.

翻译：数字病理学的加速应用与深度学习的发展，已使针对多种疾病和患者群体的各类病理学任务得以构建强大模型。然而，由于医学领域标签稀缺，模型训练常面临困难，且其应用局限于特定任务和训练所针对的疾病。此外，组织病理学中的大多数模型仅利用图像数据，这与人类相互教学和推理组织病理学实体的方式形成鲜明对比。我们提出用于组织病理学的标注对比学习（CONCH），这是一种通过任务无关预训练，利用多种来源的组织病理学图像、生物医学文本，尤其超过117万图像-文本对开发的视觉-语言基础模型。在13项多样化基准测试的评估中，CONCH可迁移至涵盖组织病理学图像和/或文本的广泛下游任务，在组织学图像分类、分割、标注、文本到图像及图像到文本检索任务中均达到最先进性能。CONCH代表着同期组织病理学视觉-语言预训练系统的重大飞跃，有望直接促进大量需要极少或无需额外监督微调的基于机器学习的工作流程。