Multimodal self-supervised representation learning has consistently proven to be a highly effective method in medical image analysis, offering strong task performance and producing biologically informed insights. However, these methods heavily rely on large, paired datasets, which is prohibitive for their use in scenarios where paired data does not exist, or there is only a small amount available. In contrast, image generation methods can work well on very small datasets, and can find mappings between unpaired datasets, meaning an effectively unlimited amount of paired synthetic data can be generated. In this work, we demonstrate that representation learning can be significantly improved by synthetically generating paired information, both compared to training on either single-modality (up to 4.4x error reduction) or authentic multi-modal paired datasets (up to 5.6x error reduction).

翻译：多模态自监督表示学习在医学图像分析中一直被证明是一种非常有效的方法，能够提供强大的任务性能并产生具有生物学意义的见解。然而，这些方法严重依赖大规模的配对数据集，这限制了它们在不存在配对数据或仅有少量可用数据场景中的应用。相比之下，图像生成方法可以在非常小的数据集上表现良好，并能找到非配对数据集之间的映射，这意味着可以生成几乎无限的配对合成数据。在这项工作中，我们证明，通过合成生成配对信息可以显著改进表示学习，与单模态训练（错误率降低高达4.4倍）或真实多模态配对数据集训练（错误率降低高达5.6倍）相比，均表现出优势。