While machine learning is currently transforming the field of histopathology, the domain lacks a comprehensive evaluation of state-of-the-art models based on essential but complementary quality requirements beyond a mere classification accuracy. In order to fill this gap, we developed a new methodology to extensively evaluate a wide range of classification models, including recent vision transformers, and convolutional neural networks such as: ConvNeXt, ResNet (BiT), Inception, ViT and Swin transformer, with and without supervised or self-supervised pretraining. We thoroughly tested the models on five widely used histopathology datasets containing whole slide images of breast, gastric, and colorectal cancer and developed a novel approach using an image-to-image translation model to assess the robustness of a cancer classification model against stain variations. Further, we extended existing interpretability methods to previously unstudied models and systematically reveal insights of the models' classifications strategies that can be transferred to future model architectures.

翻译：尽管机器学习正在深刻改变组织病理学领域，但该领域目前缺乏基于超出单纯分类准确性的关键互补质量要求对最先进模型进行全面评估。为填补这一空白，我们开发了新方法对包括近期视觉Transformer及卷积神经网络在内的广泛分类模型进行系统性评估，具体涉及ConvNeXt、ResNet (BiT)、Inception、ViT和Swin Transformer，同时涵盖有监督与自监督预训练模型。我们在包含乳腺、胃及结直肠癌全切片图像的五个常用组织病理学数据集上对模型进行了充分测试，并首创性地采用图像到图像翻译模型评估癌症分类模型对染色变异鲁棒性的方法。此外，我们将现有可解释性方法拓展至此前未被研究的模型架构，系统揭示了可迁移至未来模型架构的分类策略内在机制。