The differences among medical imaging modalities, driven by distinct underlying principles, pose significant challenges for generalization in multi-modal medical tasks. Beyond modality gaps, individual variations, such as differences in organ size and metabolic rate, further impede a model's ability to generalize effectively across both modalities and diverse populations. Despite the importance of personalization, existing approaches to multi-modal generalization often neglect individual differences, focusing solely on common anatomical features. This limitation may result in weakened generalization in various medical tasks. In this paper, we unveil that personalization is critical for multi-modal generalization. Specifically, we propose an approach to achieve personalized generalization through approximating the underlying personalized invariant representation ${X}_h$ across various modalities by leveraging individual-level constraints and a learnable biological prior. We validate the feasibility and benefits of learning a personalized ${X}_h$, showing that this representation is highly generalizable and transferable across various multi-modal medical tasks. Extensive experimental results consistently show that the additionally incorporated personalization significantly improves performance and generalization across diverse scenarios, confirming its effectiveness.

翻译：医学成像模态之间的差异，源于不同的基本原理，为多模态医学任务的泛化带来了重大挑战。除了模态差异之外，个体差异，如器官大小和代谢率的不同，进一步阻碍了模型在跨模态和跨不同人群时有效泛化的能力。尽管个性化至关重要，但现有的多模态泛化方法常常忽视个体差异，仅关注共同的解剖学特征。这种局限性可能导致模型在各种医学任务中的泛化能力减弱。在本文中，我们揭示了个人化对于多模态泛化至关重要。具体而言，我们提出了一种方法，通过利用个体层面的约束和一个可学习的生物学先验，来近似跨各种模态的底层个性化不变表示 ${X}_h$，从而实现个性化泛化。我们验证了学习个性化 ${X}_h$ 的可行性和益处，表明该表示具有高度的可泛化性和可迁移性，适用于各种多模态医学任务。大量的实验结果一致表明，额外引入的个性化显著提高了模型在多样化场景下的性能和泛化能力，证实了其有效性。