Whole slide images are the foundation of digital pathology for the diagnosis and treatment of carcinomas. Writing pathology reports is laborious and error-prone for inexperienced pathologists. To reduce the workload and improve clinical automation, we investigate how to generate pathology reports given whole slide images. On the data end, we curated the largest WSI-text dataset (PathText). In specific, we collected nearly 10000 high-quality WSI-text pairs for visual-language models by recognizing and cleaning pathology reports which narrate diagnostic slides in TCGA. On the model end, we propose the multiple instance generative model (MI-Gen) which can produce pathology reports for gigapixel WSIs. We benchmark our model on the largest subset of TCGA-PathoText. Experimental results show our model can generate pathology reports which contain multiple clinical clues and achieve competitive performance on certain slide-level tasks. We observe that simple semantic extraction from the pathology reports can achieve the best performance (0.838 of F1 score) on BRCA subtyping surpassing previous state-of-the-art approaches. Our collected dataset and related code are available.

翻译：全切片图像是数字病理学诊断和治疗癌症的基础。对于经验不足的病理学家而言，撰写病理报告既费力又容易出错。为减轻工作负担并提升临床自动化水平，我们研究了如何根据全切片图像生成病理报告。在数据方面，我们构建了最大的WSI-文本数据集（PathText）。具体而言，我们通过识别和清理TCGA中描述诊断切片的病理报告，收集了近10000个高质量WSI-文本对用于视觉语言模型。在模型方面，我们提出了多实例生成模型（MI-Gen），该模型可为千兆像素级WSI生成病理报告。我们在TCGA-PathoText的最大子集上对我们的模型进行了基准测试。实验结果表明，我们的模型能够生成包含多种临床线索的病理报告，并在某些切片级任务上取得了有竞争力的性能。我们观察到，对病理报告进行简单的语义提取即可在BRCA亚型分型任务上取得最佳性能（F1分数达0.838），超越了先前的最先进方法。我们收集的数据集及相关代码已公开。