Computer aided diagnosis (CAD) has gained an increased amount of attention in the general research community over the last years as an example of a typical limited data application - with experiments on labeled 100k-200k datasets. Although these datasets are still small compared to natural image datasets like ImageNet1k, ImageNet21k and JFT, they are large for annotated medical datasets, where 1k-10k labeled samples are much more common. There is no baseline on which methods to build on in the low data regime. In this work we bridge this gap by providing an extensive study on medical image classification with limited annotations (5k). We present a study of modern architectures applied to a fixed low data regime of 5000 images on the CheXpert dataset. Conclusively we find that models pretrained on ImageNet21k achieve a higher AUC and larger models require less training steps. All models are quite well calibrated even though we only fine-tuned on 5000 training samples. All 'modern' architectures have higher AUC than ResNet50. Regularization of Big Transfer Models with MixUp or Mean Teacher improves calibration, MixUp also improves accuracy. Vision Transformer achieve comparable or on par results to Big Transfer Models.

翻译：计算机辅助诊断作为典型的小样本数据应用场景，近年来受到学术界的广泛关注——相关实验通常基于标注规模为10万至20万的数据集。尽管这些数据集相较于ImageNet1k、ImageNet21k和JFT等自然图像数据集仍显不足，但在标注样本普遍仅有1千至1万例的医学数据领域已属大规模。目前低数据量场景下缺乏方法论基准。本研究通过系统探究小样本（5000例）医学图像分类任务填补了这一空白。我们在CheXpert数据集的固定低数据量（5000张图像）条件下，对当代多种架构进行了对比研究。最终发现：在ImageNet21k上预训练的模型可获得更高的AUC值，且大型模型所需训练步数更少。尽管仅基于5000个训练样本进行微调，所有模型仍具备良好的校准性能。所有"现代"架构的AUC值均优于ResNet50。对Big Transfer模型采用MixUp或Mean Teacher正则化可提升校准效果，其中MixUp还能改善准确率。Vision Transformer取得了与Big Transfer模型相当或持平的性能。