The foundation models based on pre-training technology have significantly advanced artificial intelligence from theoretical to practical applications. These models have facilitated the feasibility of computer-aided diagnosis for widespread use. Medical contrastive vision-language pre-training, which does not require human annotations, is an effective approach for guiding representation learning using description information in diagnostic reports. However, the effectiveness of pre-training is limited by the large-scale semantic overlap and shifting problems in medical field. To address these issues, we propose the Knowledge-Boosting Contrastive Vision-Language Pre-training framework (KoBo), which integrates clinical knowledge into the learning of vision-language semantic consistency. The framework uses an unbiased, open-set sample-wise knowledge representation to measure negative sample noise and supplement the correspondence between vision-language mutual information and clinical knowledge. Extensive experiments validate the effect of our framework on eight tasks including classification, segmentation, retrieval, and semantic relatedness, achieving comparable or better performance with the zero-shot or few-shot settings. Our code is open on https://github.com/ChenXiaoFei-CS/KoBo.

翻译：基于预训练技术的基础模型显著推动了人工智能从理论到实际应用的进展。这类模型促进了计算机辅助诊断在广泛使用中的可行性。医学对比视觉-语言预训练无需人工标注，是一种利用诊断报告中的描述信息引导表示学习的有效方法。然而，预训练的有效性受到医学领域大规模语义重叠与语义漂移问题的制约。为解决这些问题，我们提出知识增强对比视觉-语言预训练框架（KoBo），该框架将临床知识融入视觉-语言语义一致性的学习过程中。该框架采用无偏、开放集样本级知识表示，用于度量负样本噪声，并补充视觉-语言互信息与临床知识之间的对应关系。大量实验在包括分类、分割、检索和语义关联性在内的八项任务上验证了本框架的效果，在零样本或少样本设置下达到了相当或更优的性能。我们的代码已开源在 https://github.com/ChenXiaoFei-CS/KoBo。