[18F]-Fluorodeoxyglucose (FDG) positron emission tomography - computed tomography (PET-CT) has become the imaging modality of choice for diagnosing many cancers. Co-learning complementary PET-CT imaging features is a fundamental requirement for automatic tumor segmentation and for developing computer aided cancer diagnosis systems. In this study, we propose a hyper-connected transformer (HCT) network that integrates a transformer network (TN) with a hyper connected fusion for multi-modality PET-CT images. The TN was leveraged for its ability to provide global dependencies in image feature learning, which was achieved by using image patch embeddings with a self-attention mechanism to capture image-wide contextual information. We extended the single-modality definition of TN with multiple TN based branches to separately extract image features. We also introduced a hyper connected fusion to fuse the contextual and complementary image features across multiple transformers in an iterative manner. Our results with two clinical datasets show that HCT achieved better performance in segmentation accuracy when compared to the existing methods.

翻译：[18F]-氟脱氧葡萄糖（FDG）正电子发射断层扫描-计算机断层扫描（PET-CT）已成为诊断多种癌症的首选成像方式。协同学习互补的PET-CT成像特征是实现自动肿瘤分割及开发计算机辅助癌症诊断系统的基本需求。本研究提出一种超连接Transformer（HCT）网络，该网络将Transformer网络（TN）与超连接融合机制相结合，用于多模态PET-CT图像处理。TN凭借其提供图像特征学习全局依赖关系的能力被采用，通过使用图像块嵌入与自注意力机制捕获全局上下文信息实现该能力。我们将TN的单模态定义扩展为基于多个TN分支的结构，分别提取图像特征。同时引入超连接融合，以迭代方式融合来自多个Transformer的上下文与互补图像特征。在两个临床数据集上的结果表明，与现有方法相比，HCT在分割精度方面取得了更优性能。