The emerging task of fine-grained image classification in low-data regimes assumes the presence of low inter-class variance and large intra-class variation along with a highly limited amount of training samples per class. However, traditional ways of separately dealing with fine-grained categorisation and extremely scarce data may be inefficient under both these harsh conditions presented together. In this paper, we present a novel framework, called AD-Net, aiming to enhance deep neural network performance on this challenge by leveraging the power of Augmentation and Distillation techniques. Specifically, our approach is designed to refine learned features through self-distillation on augmented samples, mitigating harmful overfitting. We conduct comprehensive experiments on popular fine-grained image classification benchmarks where our AD-Net demonstrates consistent improvement over traditional fine-tuning and state-of-the-art low-data techniques. Remarkably, with the smallest data available, our framework shows an outstanding relative accuracy increase of up to 45 % compared to standard ResNet-50 and up to 27 % compared to the closest SOTA runner-up. We emphasise that our approach is practically architecture-independent and adds zero extra cost at inference time. Additionally, we provide an extensive study on the impact of every framework's component, highlighting the importance of each in achieving optimal performance. Source code and trained models are publicly available at github.com/demidovd98/fgic_lowd.

翻译：低数据场景下的细粒度图像分类这一新兴任务，面临着类间差异小、类内变化大以及每类训练样本数量极其有限的双重挑战。然而，传统分别处理细粒度分类与极端数据稀缺问题的方法，在两种严苛条件同时存在时可能效率低下。本文提出一种名为AD-Net的新型框架，旨在通过融合数据增强与知识蒸馏技术的优势，提升深度神经网络在此类挑战中的性能。具体而言，我们的方法通过对增强样本进行自蒸馏来优化学习到的特征，从而缓解有害的过拟合现象。我们在主流细粒度图像分类基准数据集上进行了全面实验，结果表明AD-Net相较于传统微调方法和当前最先进的低数据技术均能实现持续的性能提升。值得注意的是，在可用数据量最少的配置下，本框架相比标准ResNet-50实现了高达45%的相对准确率提升，相比性能最接近的现有最优方法也达到了27%的提升幅度。我们特别强调，该方法实际与网络架构无关，且在推理阶段不产生任何额外计算成本。此外，我们通过详尽的消融实验分析了框架各组件的影响，阐明了每个模块对实现最优性能的重要作用。源代码及训练模型已在github.com/demidovd98/fgic_lowd公开。