Whole slide image (WSI) has been widely used to assist automated diagnosis under the deep learning fields. However, most previous works only discuss the SINGLE task setting which is not aligned with real clinical setting, where pathologists often conduct multiple diagnosis tasks simultaneously. Also, it is commonly recognized that the multi-task learning paradigm can improve learning efficiency by exploiting commonalities and differences across multiple tasks. To this end, we present a novel multi-task framework (i.e., MulGT) for WSI analysis by the specially designed Graph-Transformer equipped with Task-aware Knowledge Injection and Domain Knowledge-driven Graph Pooling modules. Basically, with the Graph Neural Network and Transformer as the building commons, our framework is able to learn task-agnostic low-level local information as well as task-specific high-level global representation. Considering that different tasks in WSI analysis depend on different features and properties, we also design a novel Task-aware Knowledge Injection module to transfer the task-shared graph embedding into task-specific feature spaces to learn more accurate representation for different tasks. Further, we elaborately design a novel Domain Knowledge-driven Graph Pooling module for each task to improve both the accuracy and robustness of different tasks by leveraging different diagnosis patterns of multiple tasks. We evaluated our method on two public WSI datasets from TCGA projects, i.e., esophageal carcinoma and kidney carcinoma. Experimental results show that our method outperforms single-task counterparts and the state-of-theart methods on both tumor typing and staging tasks.

翻译：全切片图像（Whole Slide Image, WSI）在深度学习领域已被广泛用于辅助自动诊断。然而，以往大多数工作仅探讨单任务设置，这与实际临床场景不符——病理学家通常需要同时执行多项诊断任务。此外，学界普遍认为多任务学习范式能够通过挖掘多个任务之间的共性与差异来提升学习效率。为此，我们提出了一种专为WSI分析设计的新型多任务框架（即MulGT），该框架采用特制的图-Transformer，并配备任务感知知识注入模块与领域知识驱动图池化模块。基于图神经网络和Transformer作为基础构建单元，我们的框架既能学习与任务无关的底层局部信息，也能学习与任务相关的高层全局表征。针对WSI分析中不同任务依赖不同特征与属性的特点，我们进一步设计了新颖的任务感知知识注入模块，将任务共享的图嵌入映射到任务特定的特征空间，从而为不同任务学习更精确的表征。此外，我们为每个任务精心设计了领域知识驱动图池化模块，通过利用多任务间不同的诊断模式来提升各任务的准确性与鲁棒性。我们在TCGA项目的两个公开WSI数据集（食管癌与肾癌）上评估了该方法。实验结果表明，在肿瘤分型与分期任务上，我们的方法均优于单任务对比方法和当前最先进方法。