Whole slide image (WSI) has been widely used to assist automated diagnosis under the deep learning fields. However, most previous works only discuss the SINGLE task setting which is not aligned with real clinical setting, where pathologists often conduct multiple diagnosis tasks simultaneously. Also, it is commonly recognized that the multi-task learning paradigm can improve learning efficiency by exploiting commonalities and differences across multiple tasks. To this end, we present a novel multi-task framework (i.e., MulGT) for WSI analysis by the specially designed Graph-Transformer equipped with Task-aware Knowledge Injection and Domain Knowledge-driven Graph Pooling modules. Basically, with the Graph Neural Network and Transformer as the building commons, our framework is able to learn task-agnostic low-level local information as well as task-specific high-level global representation. Considering that different tasks in WSI analysis depend on different features and properties, we also design a novel Task-aware Knowledge Injection module to transfer the task-shared graph embedding into task-specific feature spaces to learn more accurate representation for different tasks. Further, we elaborately design a novel Domain Knowledge-driven Graph Pooling module for each task to improve both the accuracy and robustness of different tasks by leveraging different diagnosis patterns of multiple tasks. We evaluated our method on two public WSI datasets from TCGA projects, i.e., esophageal carcinoma and kidney carcinoma. Experimental results show that our method outperforms single-task counterparts and the state-of-theart methods on both tumor typing and staging tasks.

翻译：全切片图像（WSI）已被广泛应用于深度学习领域的自动诊断辅助。然而，现有研究大多仅讨论单一任务设置，这与实际临床场景不符——病理学家通常需同时执行多项诊断任务。同时，多任务学习范式通过利用不同任务间的共性与差异，能够有效提升学习效率。为此，我们提出了一种面向WSI分析的新型多任务框架（即MulGT），该框架采用专门设计的图-Transformer，并配备任务感知知识注入与领域知识驱动图池化模块。基于图神经网络和Transformer作为基础架构，本框架能够学习任务无关的低层局部信息以及任务特定的高层全局表征。考虑到WSI分析中不同任务依赖不同的特征与属性，我们进一步设计了新颖的任务感知知识注入模块，将任务共享的图嵌入映射至任务特定的特征空间，从而为不同任务学习更精准的表征。此外，我们针对每个任务精心设计了领域知识驱动图池化模块，通过利用不同任务的诊断模式差异，提升各任务的准确性与鲁棒性。我们在TCGA项目中的两个公开WSI数据集（食管癌与肾癌）上进行了评估。实验结果表明，该方法在肿瘤分型与分期任务中均优于单任务对照方法及当前最优方法。