Multi-task learning (MTL) is a powerful approach in deep learning that leverages the information from multiple tasks during training to improve model performance. In medical imaging, MTL has shown great potential to solve various tasks. However, existing MTL architectures in medical imaging are limited in sharing information across tasks, reducing the potential performance improvements of MTL. In this study, we introduce a novel attention-based MTL framework to better leverage inter-task interactions for various tasks from pixel-level to image-level predictions. Specifically, we propose a Cross-Task Attention Network (CTAN) which utilizes cross-task attention mechanisms to incorporate information by interacting across tasks. We validated CTAN on four medical imaging datasets that span different domains and tasks including: radiation treatment planning prediction using planning CT images of two different target cancers (Prostate, OpenKBP); pigmented skin lesion segmentation and diagnosis using dermatoscopic images (HAM10000); and COVID-19 diagnosis and severity prediction using chest CT scans (STOIC). Our study demonstrates the effectiveness of CTAN in improving the accuracy of medical imaging tasks. Compared to standard single-task learning (STL), CTAN demonstrated a 4.67% improvement in performance and outperformed both widely used MTL baselines: hard parameter sharing (HPS) with an average performance improvement of 3.22%; and multi-task attention network (MTAN) with a relative decrease of 5.38%. These findings highlight the significance of our proposed MTL framework in solving medical imaging tasks and its potential to improve their accuracy across domains.

翻译：多任务学习（MTL）是深度学习中的一种强大方法，通过利用训练过程中多个任务的信息来提升模型性能。在医学影像领域，MTL在解决各类任务方面展现出巨大潜力。然而，现有医学影像中的MTL架构在跨任务信息共享方面存在局限，削弱了MTL的潜在性能提升。本研究提出一种基于注意力的新型MTL框架，以更有效地利用跨任务交互信息，适用于从像素级到图像级预测的各类任务。具体而言，我们提出了跨任务注意力网络（CTAN），该网络通过跨任务注意力机制，在任务间交互并融合信息。我们在涵盖不同领域和任务的四个医学影像数据集上验证了CTAN，包括：利用规划CT图像进行放射治疗规划预测（针对两种不同目标癌症：前列腺癌、OpenKBP）；利用皮肤镜图像进行色素性皮肤病变分割与诊断（HAM10000）；以及利用胸部CT扫描进行COVID-19诊断与严重程度预测（STOIC）。我们的研究表明，CTAN在提升医学影像任务准确性方面具有显著效果。与标准单任务学习（STL）相比，CTAN实现了4.67%的性能提升；同时优于广泛使用的MTL基线方法：硬参数共享（HPS）平均性能提升3.22%，而多任务注意力网络（MTAN）相对降低5.38%。这些发现凸显了我们提出的MTL框架在解决医学影像任务中的重要性及其跨领域提升任务准确率的潜力。