Data is the foundation for the development of computer vision, and the establishment of datasets plays an important role in advancing the techniques of fine-grained visual categorization~(FGVC). In the existing FGVC datasets used in computer vision, it is generally assumed that each collected instance has fixed characteristics and the distribution of different categories is relatively balanced. In contrast, the real world scenario reveals the fact that the characteristics of instances tend to vary with time and exhibit a long-tailed distribution. Hence, the collected datasets may mislead the optimization of the fine-grained classifiers, resulting in unpleasant performance in real applications. Starting from the real-world conditions and to promote the practical progress of fine-grained visual categorization, we present a Concept Drift and Long-Tailed Distribution dataset. Specifically, the dataset is collected by gathering 11195 images of 250 instances in different species for 47 consecutive months in their natural contexts. The collection process involves dozens of crowd workers for photographing and domain experts for labeling. Meanwhile, we propose a feature recombination framework to address the learning challenges associated with CDLT. Experimental results validate the efficacy of our method while also highlighting the limitations of popular large vision-language models (e.g., CLIP) in the context of long-tailed distributions. This emphasizes the significance of CDLT as a benchmark for investigating these challenges.
翻译:数据是计算机视觉发展的基础,而数据集的建立对推进细粒度视觉分类(FGVC)技术具有重要作用。在计算机视觉领域现有的FGVC数据集中,通常假设每个采集的实例具有固定特征,且不同类别的分布相对均衡。然而,现实世界场景表明实例特征往往随时间变化,并呈现长尾分布特性。因此,现有数据集可能误导细粒度分类器的优化,导致在实际应用中性能欠佳。基于现实条件并推动细粒度视觉分类的实际进展,我们提出了概念漂移与长尾分布数据集。具体而言,该数据集通过连续47个月在自然环境中采集250个不同物种实例的11195张图像构建而成。采集过程涉及数十名众包拍摄人员和领域专家进行标注。同时,我们提出了一种特征重组框架以应对CDLT相关的学习挑战。实验结果验证了我们方法的有效性,同时也凸显了流行的大型视觉语言模型(如CLIP)在长尾分布场景中的局限性。这进一步强调了CDLT作为研究这些挑战的基准数据集的重要意义。