Low-resource settings are well-established in natural language processing, where many languages lack sufficient data for machine learning at scale. However, low-resource problems are under-explored in computer vision. In this paper, we strive to address this gap and explore the challenges of low-resource image tasks with vision foundation models. Thus, we first collect a benchmark of genuinely low-resource image data, covering historic maps, circuit diagrams, and mechanical drawings. These low-resource settings all share the three challenges of data scarcity, fine-grained differences, and the distribution shift from natural images to the specialized domain of interest. While existing foundation models have shown impressive generalizability, we find they cannot transfer well to our low-resource tasks. To begin to tackle the challenges of low-resource vision, we introduce one simple baseline per challenge. Specifically, we propose to i) enlarge the data space by generative models, ii) adopt the best sub-kernels to encode local regions for fine-grained difference discovery and iii) learn attention for specialized domains. Experiments on the three low-resource data sources in our benchmark demonstrate our proposals already provide a better baseline than common transfer learning, data augmentation, and fine-grained methods. This highlights the unique characteristics and challenges of low-resource vision for foundation models that warrant further investigation. Project website: https://xiaobai1217.github.io/Low-Resource-Vision/.

翻译：低资源设定在自然语言处理中已得到充分研究，其中许多语言缺乏大规模机器学习所需的数据。然而，计算机视觉领域的低资源问题尚未充分探索。本文致力于弥补这一空白，探索基于视觉基础模型处理低资源图像任务所面临的挑战。为此，我们首先收集了一个涵盖历史地图、电路图和机械图纸的真实低资源图像数据基准。这些低资源设定均面临数据稀缺、细粒度差异以及自然图像与专业领域分布偏移三大挑战。尽管现有基础模型展现出显著的泛化能力，但我们发现它们无法很好地迁移至我们的低资源任务。为初步应对低资源视觉的挑战，我们针对每个挑战提出一个简单基线。具体而言，我们建议：i) 通过生成模型扩展数据空间；ii) 采用最优子内核编码局部区域以发现细粒度差异；iii) 学习针对专业领域的注意力机制。在基准中三个低资源数据源上的实验表明，我们的提议已提供比常见迁移学习、数据增强和细粒度方法更优的基线。这凸显了基础模型在低资源视觉中需进一步研究的独特特征与挑战。项目网站：https://xiaobai1217.github.io/Low-Resource-Vision/。