Despite being resource-intensive to train, 3D convolutional neural networks (CNNs) have been the standard approach to classify CT and MRI scans. Recent work suggests that deep multiple instance learning (MIL) may be a more efficient alternative for 3D brain scans, especially when the pre-trained image encoder used to embed each 2D slice is frozen and only the pooling operation and classifier are trained. In this paper, we provide a systematic comparison of simple MIL, attention-based MIL, 3D CNNs, and 3D ViTs across three CT and four MRI datasets, including two large datasets of at least 10,000 scans. Our goal is to help resource-constrained practitioners understand which neural networks work well for 3D neuroimages and why. We further compare design choices for attention-based MIL, including different encoders, pooling operations, and architectural orderings. We find that simple mean pooling MIL, without any learnable attention, matches or outperforms recent MIL or 3D CNN alternatives on 4 of 6 moderate-sized tasks. This baseline remains competitive on two large datasets while being 25x faster to train. To explain mean pooling's success, we examine per-slice attention quality and a semi-synthetic dataset where we can derive the best possible classifier via a Bayes estimator. This analysis reveals the limits of existing MIL approaches and suggests routes for future improvements.

翻译：尽管训练成本高昂，三维卷积神经网络（3D CNN）仍是CT和MRI影像分类的标准方法。近期研究表明，深度多实例学习（MIL）可能是处理三维脑部扫描的更高效替代方案，尤其当用于嵌入每个二维切片的预训练图像编码器被冻结，仅训练池化操作与分类器时。本文在三个CT数据集和四个MRI数据集（含两个至少包含10,000次扫描的大型数据集）上，系统比较了简单MIL、基于注意力的MIL、3D CNN及3D ViT。我们的目标在于帮助资源受限的研究者理解哪些神经网络适用于三维神经影像及其原因。进一步地，我们对比了基于注意力的MIL的设计选择，包括不同编码器、池化操作及架构顺序。研究发现，在6个中等规模任务中的4个任务上，无需可学习注意力的简单均值池化MIL可达到甚至超越现有MIL或3D CNN方法的性能。该基线方法在两个大型数据集上仍具竞争力，且训练速度提升了25倍。为解释均值池化的成功机理，我们分析了逐切片注意力质量，并构造了一个可通过贝叶斯估计推导最优分类器的半合成数据集。该分析揭示了现有MIL方法的局限性，并为未来改进提供了方向。