Medical image analysis frequently encounters data scarcity challenges. Transfer learning has been effective in addressing this issue while conserving computational resources. The recent advent of foundational models like the DINOv2, which uses the vision transformer architecture, has opened new opportunities in the field and gathered significant interest. However, DINOv2's performance on clinical data still needs to be verified. In this paper, we performed a glioma grading task using three clinical modalities of brain MRI data. We compared the performance of various pre-trained deep learning models, including those based on ImageNet and DINOv2, in a transfer learning context. Our focus was on understanding the impact of the freezing mechanism on performance. We also validated our findings on three other types of public datasets: chest radiography, fundus radiography, and dermoscopy. Our findings indicate that in our clinical dataset, DINOv2's performance was not as strong as ImageNet-based pre-trained models, whereas in public datasets, DINOv2 generally outperformed other models, especially when using the frozen mechanism. Similar performance was observed with various sizes of DINOv2 models across different tasks. In summary, DINOv2 is viable for medical image classification tasks, particularly with data resembling natural images. However, its effectiveness may vary with data that significantly differs from natural images such as MRI. In addition, employing smaller versions of the model can be adequate for medical task, offering resource-saving benefits. Our codes are available at https://github.com/GuanghuiFU/medical_DINOv2_eval.

翻译：医学图像分析常面临数据稀缺的挑战。迁移学习在解决该问题并节省计算资源方面表现有效。近期，基于视觉Transformer架构的DINOv2等基础模型的问世为该领域带来了新机遇并引发了广泛关注。然而，DINOv2在临床数据上的性能仍有待验证。本文利用三种临床模态的脑部MRI数据执行胶质瘤分级任务，比较了包括基于ImageNet和DINOv2的多种预训练深度学习模型在迁移学习场景中的表现，重点关注冻结机制对性能的影响。我们还在三类其他公共数据集（胸部X光片、眼底影像和皮肤镜图像）上验证了研究结论。结果表明，在临床数据集中，DINOv2的表现不及基于ImageNet的预训练模型；而在公共数据集中，尤其是采用冻结机制时，DINOv2通常优于其他模型。不同规模的DINOv2模型在不同任务中展现出相似的性能。综上所述，DINOv2适用于医学图像分类任务，尤其是在数据特征与自然图像相似时。但对于MRI等与自然图像差异显著的数据，其有效性可能有所差异。此外，采用小规模模型即可满足医学任务需求，具有资源节约优势。我们的代码开源地址为：https://github.com/GuanghuiFU/medical_DINOv2_eval。