Meta-learning has recently been an emerging data-efficient learning technique for various medical imaging operations and has helped advance contemporary deep learning models. Furthermore, meta-learning enhances the knowledge generalization of the imaging tasks by learning both shared and discriminative weights for various configurations of imaging tasks. However, existing meta-learning models attempt to learn a single set of weight initializations of a neural network that might be restrictive for multimodal data. This work aims to develop a multimodal meta-learning model for image reconstruction, which augments meta-learning with evolutionary capabilities to encompass diverse acquisition settings of multimodal data. Our proposed model called KM-MAML (Kernel Modulation-based Multimodal Meta-Learning), has hypernetworks that evolve to generate mode-specific weights. These weights provide the mode-specific inductive bias for multiple modes by re-calibrating each kernel of the base network for image reconstruction via a low-rank kernel modulation operation. We incorporate gradient-based meta-learning (GBML) in the contextual space to update the weights of the hypernetworks for different modes. The hypernetworks and the reconstruction network in the GBML setting provide discriminative mode-specific features and low-level image features, respectively. Experiments on multi-contrast MRI reconstruction show that our model, (i) exhibits superior reconstruction performance over joint training, other meta-learning methods, and context-specific MRI reconstruction methods, and (ii) better adaptation capabilities with improvement margins of 0.5 dB in PSNR and 0.01 in SSIM. Besides, a representation analysis with U-Net shows that kernel modulation infuses 80% of mode-specific representation changes in the high-resolution layers. Our source code is available at https://github.com/sriprabhar/KM-MAML/.

翻译：元学习近年来已成为多种医学影像操作中新兴的数据高效学习技术，有效推动了当代深度学习模型的发展。此外，元学习通过为不同成像任务配置学习共享权重与判别性权重，增强了成像任务的知识泛化能力。然而，现有元学习模型试图学习神经网络的单一权重初始化集合，这可能对多模态数据存在局限性。本研究旨在开发一种用于图像重建的多模态元学习模型，该模型通过引入进化能力增强元学习，以涵盖多模态数据的多样化采集设置。我们提出的模型名为KM-MAML（基于核调制的多模态元学习），其超网络可进化生成模态特定权重。这些权重通过低秩核调制操作对基网络的每个核进行重新校准，从而为多种模态提供模态特定的归纳偏置。我们在上下文空间中采用基于梯度的元学习（GBML）来更新不同模态下超网络的权重。在GBML框架中，超网络与重建网络分别提供判别性模态特定特征与低级图像特征。在多对比度MRI重建实验表明，我们的模型：（i）相较于联合训练、其他元学习方法及特定上下文的MRI重建方法，展现出更优越的重建性能；（ii）具备更优的自适应能力，PSNR提升0.5 dB，SSIM提升0.01。此外，基于U-Net的表征分析显示，核调制在高分辨率层中引入了80%的模态特定表征变化。我们的源代码已开源至：https://github.com/sriprabhar/KM-MAML/。