Whole slide image (WSI) classification is a critical task in computational pathology, requiring the processing of gigapixel-sized images, which is challenging for current deep-learning methods. Current state of the art methods are based on multi-instance learning schemes (MIL), which usually rely on pretrained features to represent the instances. Due to the lack of task-specific annotated data, these features are either obtained from well-established backbones on natural images, or, more recently from self-supervised models pretrained on histopathology. However, both approaches yield task-agnostic features, resulting in performance loss compared to the appropriate task-related supervision, if available. In this paper, we show that when task-specific annotations are limited, we can inject such supervision into downstream task training, to reduce the gap between fully task-tuned and task agnostic features. We propose Prompt-MIL, an MIL framework that integrates prompts into WSI classification. Prompt-MIL adopts a prompt tuning mechanism, where only a small fraction of parameters calibrates the pretrained features to encode task-specific information, rather than the conventional full fine-tuning approaches. Extensive experiments on three WSI datasets, TCGA-BRCA, TCGA-CRC, and BRIGHT, demonstrate the superiority of Prompt-MIL over conventional MIL methods, achieving a relative improvement of 1.49%-4.03% in accuracy and 0.25%-8.97% in AUROC while using fewer than 0.3% additional parameters. Compared to conventional full fine-tuning approaches, we fine-tune less than 1.3% of the parameters, yet achieve a relative improvement of 1.29%-13.61% in accuracy and 3.22%-27.18% in AUROC and reduce GPU memory consumption by 38%-45% while training 21%-27% faster. Our code is available at https://github.com/cvlab-stonybrook/PromptMIL.

翻译：全切片图像（WSI）分类是计算病理学中的关键任务，需处理千兆像素级图像，这对当前深度学习方法构成挑战。现有最优方法基于多实例学习（MIL）方案，通常依赖预训练特征表示实例。由于缺乏任务特定标注数据，这些特征或来自自然图像上的成熟骨干网络，或最近采用组织病理学预训练的自监督模型。然而，两种方法均生成任务无关特征，导致与可用任务监督相比的性能损失。本文表明，当任务特定标注有限时，可将此类监督注入下游任务训练，以缩小完全任务调优与任务无关特征之间的差距。我们提出Prompt-MIL，一种将提示机制集成至WSI分类的MIL框架。Prompt-MIL采用提示调优机制，仅需调节少量参数即可校准预训练特征以编码任务特定信息，而非传统全参数微调。在TCGA-BRCA、TCGA-CRC和BRIGHT三个WSI数据集上的大量实验表明，Prompt-MIL优于传统MIL方法：在额外参数少于0.3%的情况下，相对提升准确率1.49%-4.03%和AUROC 0.25%-8.97%。与传统全参数微调方法相比，我们仅微调少于1.3%的参数，却实现准确率相对提升1.29%-13.61%和AUROC相对提升3.22%-27.18%，同时训练速度加快21%-27%，GPU内存消耗降低38%-45%。代码已开源：https://github.com/cvlab-stonybrook/PromptMIL。