Background: Deep learning (DL) can extract predictive and prognostic biomarkers from routine pathology slides in colorectal cancer. For example, a DL test for the diagnosis of microsatellite instability (MSI) in CRC has been approved in 2022. Current approaches rely on convolutional neural networks (CNNs). Transformer networks are outperforming CNNs and are replacing them in many applications, but have not been used for biomarker prediction in cancer at a large scale. In addition, most DL approaches have been trained on small patient cohorts, which limits their clinical utility. Methods: In this study, we developed a new fully transformer-based pipeline for end-to-end biomarker prediction from pathology slides. We combine a pre-trained transformer encoder and a transformer network for patch aggregation, capable of yielding single and multi-target prediction at patient level. We train our pipeline on over 9,000 patients from 10 colorectal cancer cohorts. Results: A fully transformer-based approach massively improves the performance, generalizability, data efficiency, and interpretability as compared with current state-of-the-art algorithms. After training on a large multicenter cohort, we achieve a sensitivity of 0.97 with a negative predictive value of 0.99 for MSI prediction on surgical resection specimens. We demonstrate for the first time that resection specimen-only training reaches clinical-grade performance on endoscopic biopsy tissue, solving a long-standing diagnostic problem. Interpretation: A fully transformer-based end-to-end pipeline trained on thousands of pathology slides yields clinical-grade performance for biomarker prediction on surgical resections and biopsies. Our new methods are freely available under an open source license.

翻译：背景：深度学习可从结直肠癌常规病理切片中提取预测性和预后性生物标志物。例如，一种用于结直肠癌微卫星不稳定性诊断的深度学习检测方法已于2022年获批。当前方法依赖卷积神经网络，而Transformer网络在性能上超越CNN并在众多应用中逐步取代后者，但尚未大规模用于癌症生物标志物预测。此外，多数深度学习方法基于小规模患者队列训练，这限制了其临床实用性。方法：本研究开发了一种全新的全Transformer流水线，用于病理切片端到端生物标志物预测。我们结合预训练Transformer编码器与用于图像块聚合的Transformer网络，可在患者层面实现单目标及多目标预测，并在涵盖10个结直肠癌队列的9000余例患者数据上完成训练。结果：与当前最优算法相比，全Transformer方法在性能、泛化能力、数据效率及可解释性方面均实现大幅提升。在大规模多中心队列训练后，该模型对手术切除标本的微卫星不稳定性预测灵敏度达0.97，阴性预测值达0.99。我们首次证明仅基于手术切除标本训练即可在内镜活检组织上达到临床级性能，解决了长期存在的诊断难题。解读：基于数千张病理切片训练的全Transformer端到端流水线可在手术切除和活检标本中实现临床级生物标志物预测。本方法以开源协议免费提供。