Multi-sequence MRI is valuable in clinical settings for reliable diagnosis and treatment prognosis, but some sequences may be unusable or missing for various reasons. To address this issue, MRI synthesis is a potential solution. Recent deep learning-based methods have achieved good performance in combining multiple available sequences for missing sequence synthesis. Despite their success, these methods lack the ability to quantify the contributions of different input sequences and estimate the quality of generated images, making it hard to be practical. Hence, we propose an explainable task-specific synthesis network, which adapts weights automatically for specific sequence generation tasks and provides interpretability and reliability from two sides: (1) visualize the contribution of each input sequence in the fusion stage by a trainable task-specific weighted average module; (2) highlight the area the network tried to refine during synthesizing by a task-specific attention module. We conduct experiments on the BraTS2021 dataset of 1251 subjects, and results on arbitrary sequence synthesis indicate that the proposed method achieves better performance than the state-of-the-art methods. Our code is available at \url{https://github.com/fiy2W/mri_seq2seq}.
翻译:多序列MRI在临床环境中对于可靠的诊断和治疗预后具有重要意义,但部分序列可能因各种原因无法使用或缺失。为解决这一问题,MRI合成是一种潜在的解决方案。近年来,基于深度学习方法在组合多个可用序列以合成缺失序列方面取得了良好性能。尽管取得了成功,但现有方法缺乏量化不同输入序列贡献和评估生成图像质量的能力,这限制了其实用性。因此,我们提出一种可解释的任务特定合成网络,该网络能够针对特定序列生成任务自动调整权重,并从两方面提供可解释性和可靠性:(1) 通过可训练的任务特定加权平均模块,在融合阶段可视化每个输入序列的贡献;(2) 通过任务特定注意力模块,突出显示网络在合成过程中尝试优化的区域。我们在包含1251个样本的BraTS2021数据集上进行了实验,对任意序列合成任务的结果表明,所提方法优于现有最先进方法。我们的代码已开源在 \url{https://github.com/fiy2W/mri_seq2seq}。