Magnetic resonance imaging (MRI) is highly sensitive for lesion detection in the breasts. Sequences obtained with different settings can capture the specific characteristics of lesions. Such multi-parameter MRI information has been shown to improve radiologist performance in lesion classification, as well as improving the performance of artificial intelligence models in various tasks. However, obtaining multi-parameter MRI makes the examination costly in both financial and time perspectives, and there may be safety concerns for special populations, thus making acquisition of the full spectrum of MRI sequences less durable. In this study, different than naive input fusion or feature concatenation from existing MRI parameters, a novel $\textbf{I}$ntegrated MRI $\textbf{M}$ulti-$\textbf{P}$arameter reinf$\textbf{O}$rcement fusion generato$\textbf{R}$ wi$\textbf{T}$h $\textbf{A}$tte$\textbf{NT}$ion Network (IMPORTANT-Net) is developed to generate missing parameters. First, the parameter reconstruction module is used to encode and restore the existing MRI parameters to obtain the corresponding latent representation information at any scale level. Then the multi-parameter fusion with attention module enables the interaction of the encoded information from different parameters through a set of algorithmic strategies, and applies different weights to the information through the attention mechanism after information fusion to obtain refined representation information. Finally, a reinforcement fusion scheme embedded in a $V^{-}$-shape generation module is used to combine the hierarchical representations to generate the missing MRI parameter. Results showed that our IMPORTANT-Net is capable of generating missing MRI parameters and outperforms comparable state-of-the-art networks. Our code is available at https://github.com/Netherlands-Cancer-Institute/MRI_IMPORTANT_NET.
翻译:摘要:磁共振成像(MRI)对乳腺病变检测具有高度灵敏性。通过不同成像参数获取的序列能捕捉病变的特定特征。这种多参数MRI信息已被证明可提升放射科医生在病变分类中的表现,并改善人工智能模型在各类任务中的性能。然而,获取多参数MRI会导致检查在金钱和时间成本上均较为昂贵,且对特殊人群可能存在安全性问题,因此难以实现全套MRI序列的持续采集。本研究不同于简单地对现有MRI参数进行输入融合或特征拼接,提出了一种新型【整合多参数MRI强化融合与注意力网络】(IMPORTANT-Net)用于生成缺失参数。首先,利用参数重建模块对现有MRI参数进行编码与重构,以获取任意尺度下的对应潜在表征信息。随后,多参数融合注意力模块通过一系列算法策略实现不同参数编码信息的交互,并在信息融合后通过注意力机制赋予不同权重以获取精细化表征信息。最后,嵌入V型生成模块的强化融合方案被用于组合分层表征,从而生成缺失的MRI参数。结果表明,我们的IMPORTANT-Net能够生成缺失的MRI参数,其性能优于可比的现有最优网络。我们的代码已开源至https://github.com/Netherlands-Cancer-Institute/MRI_IMPORTANT_NET。