The development of multi-modal medical foundation models has attracted significant attention in the field of medicine and healthcare due to their promising prospects in various clinical applications. One area of focus in this research direction is the extractions of features at different scales. While previous studies have explored feature learning at individual scales, investigation on integrating the diverse scales and modalities of information is lacking, which may hinder the potential for mutual reinforcement among these features. This paper aims to bridge this gap by proposing a method that effectively exploits multi-scale and cross-modality information to enhance the performance of medical foundation models. The proposed method simultaneously exploit features at the local, instance, modality and global aspects, facilitating comprehensive representation learning within the models. We evaluate the effectiveness of the proposed method on six open-source datasets across different clinical tasks, demonstrating its ability to enhance the performance of medical foundation models.

翻译：多模态医学基础模型的发展因其在多种临床应用中具有广阔前景，已引起医学和医疗领域的高度关注。该研究方向的一个重点是在不同尺度上进行特征提取。尽管已有研究探索了单一尺度的特征学习，但针对如何整合不同尺度和模态信息的研究仍显不足，这可能阻碍这些特征之间相互增强的潜力。本文旨在弥补这一不足，提出一种有效利用多尺度与跨模态信息以提升医学基础模型性能的方法。所提出的方法同时利用局部、实例、模态和全局四个方面的特征，促进模型内部学习全面的特征表征。我们通过六个开源数据集在多种临床任务上评估了该方法的有效性，实验结果证明其能够增强医学基础模型的性能。