Image segmentation, real-value prediction, and cross-modal translation are critical challenges in medical imaging. In this study, we propose a versatile multi-task neural network framework, based on an enhanced Transformer U-Net architecture, capable of simultaneously, selectively, and adaptively addressing these medical image tasks. Validation is performed on a public repository of human brain MR and CT images. We decompose the traditional problem of synthesizing CT images into distinct subtasks, which include skull segmentation, Hounsfield unit (HU) value prediction, and image sequential reconstruction. To enhance the framework's versatility in handling multi-modal data, we expand the model with multiple image channels. Comparisons between synthesized CT images derived from T1-weighted and T2-Flair images were conducted, evaluating the model's capability to integrate multi-modal information from both morphological and pixel value perspectives.
翻译:图像分割、实值预测与跨模态转换是医学影像中的关键挑战。本研究提出一种基于增强型Transformer U-Net架构的多功能多任务神经网络框架,能够同步、选择性和自适应地处理这些医学图像任务。我们在公开的人脑MR与CT图像数据集上进行了验证。将传统的CT图像合成问题分解为多个子任务,包括颅骨分割、亨氏单位(HU)值预测以及图像序列重建。为增强框架处理多模态数据的适应性,我们通过多图像通道扩展了模型。基于T1加权和T2-Flair图像合成CT图像的比较实验,从形态学和像素值两个角度评估了模型整合多模态信息的能力。