Medical segmentation has grown exponentially through the advent of a fully convolutional network (FCN), and we have now reached a turning point through the success of Transformer. However, the different characteristics of the modality have not been fully integrated into Transformer for medical segmentation. In this work, we propose the novel hybrid fusion Transformer (HFTrans) for multisequence MRI image segmentation. We take advantage of the differences among multimodal MRI sequences and utilize the Transformer layers to integrate the features extracted from each modality as well as the features of the early fused modalities. We validate the effectiveness of our hybrid-fusion method in three-dimensional (3D) medical segmentation. Experiments on two public datasets, BraTS2020 and MRBrainS18, show that the proposed method outperforms previous state-of-the-art methods on the task of brain tumor segmentation and brain structure segmentation.

翻译：医学分割随着全卷积网络（FCN）的出现呈指数级增长，而如今通过Transformer的成功，我们已到达了一个转折点。然而，模态的不同特性尚未被完全整合到用于医学分割的Transformer中。在这项工作中，我们提出了一种新颖的混合融合Transformer（HFTrans），用于多序列MRI图像分割。我们利用多模态MRI序列之间的差异，并借助Transformer层整合从每种模态提取的特征以及早期融合模态的特征。我们在三维（3D）医学分割中验证了混合融合方法的有效性。在BraTS2020和MRBrainS18两个公开数据集上的实验表明，所提出的方法在脑肿瘤分割和脑结构分割任务上优于先前的最先进方法。