The foundation models based on pre-training technology have significantly advanced artificial intelligence from theoretical to practical applications. These models have facilitated the feasibility of computer-aided diagnosis for widespread use. Medical contrastive vision-language pre-training, which does not require human annotations, is an effective approach for guiding representation learning using description information in diagnostic reports. However, the effectiveness of pre-training is limited by the large-scale semantic overlap and shifting problems in medical field. To address these issues, we propose the Knowledge-Boosting Contrastive Vision-Language Pre-training framework (KoBo), which integrates clinical knowledge into the learning of vision-language semantic consistency. The framework uses an unbiased, open-set sample-wise knowledge representation to measure negative sample noise and supplement the correspondence between vision-language mutual information and clinical knowledge. Extensive experiments validate the effect of our framework on eight tasks including classification, segmentation, retrieval, and semantic relatedness, achieving comparable or better performance with the zero-shot or few-shot settings. Our code is open on https://github.com/ChenXiaoFei-CS/KoBo.

翻译：基于预训练技术的基础模型显著推动了人工智能从理论走向实际应用。这些模型促进了计算机辅助诊断在广泛场景中的可行性。医学对比视觉-语言预训练无需人工标注，是利用诊断报告中的描述信息指导表示学习的有效方法。然而，预训练的效果受到医学领域大规模语义重叠和语义漂移问题的限制。为解决这些问题，我们提出了知识增强对比视觉-语言预训练框架（KoBo），该框架将临床知识融入视觉-语言语义一致性的学习中。该框架利用无偏、开放集样本级知识表示来度量负样本噪声，并补充视觉-语言互信息与临床知识之间的对应关系。大量实验在包括分类、分割、检索和语义相关性等八项任务上验证了我们的框架效果，在零样本或少样本设置下实现了可比或更优的性能。我们的代码已开源在 https://github.com/ChenXiaoFei-CS/KoBo。