Low-resource settings are well-established in natural language processing, where many languages lack sufficient data for deep learning at scale. However, low-resource problems are under-explored in computer vision. In this paper, we address this gap and explore the challenges of low-resource image tasks with vision foundation models. We first collect a benchmark of genuinely low-resource image data, covering historic maps, circuit diagrams, and mechanical drawings. These low-resource settings all share three challenges: data scarcity, fine-grained differences, and the distribution shift from natural images to the specialized domain of interest. While existing foundation models have shown impressive generalizability, we find they cannot transfer well to our low-resource tasks. To begin to tackle the challenges of low-resource vision, we introduce one simple baseline per challenge. Specifically, we i) enlarge the data space by generative models, ii) adopt the best sub-kernels to encode local regions for fine-grained difference discovery and iii) learn attention for specialized domains. Experiments on our three low-resource tasks demonstrate our proposals already provide a better baseline than transfer learning, data augmentation, and fine-grained methods. This highlights the unique characteristics and challenges of low-resource vision for foundation models that warrant further investigation. Project page: https://xiaobai1217.github.io/Low-Resource-Vision/.

翻译：低资源环境在自然语言处理中已有充分研究，其中许多语言缺乏大规模深度学习所需的数据。然而，低资源问题在计算机视觉领域尚未得到充分探索。本文填补了这一空白，探讨了基于视觉基础模型的低资源图像任务挑战。我们首先构建了一个真正的低资源图像数据基准，涵盖历史地图、电路图和机械图纸。这些低资源设置均面临三大挑战：数据稀缺、细粒度差异以及从自然图像到特定专业领域的分布偏移。尽管现有基础模型展现出令人印象深刻的泛化能力，但我们发现它们无法很好地迁移到低资源任务中。为初步解决低资源视觉的挑战，我们针对每个挑战引入了一个简单基线。具体而言，我们：i) 通过生成模型扩大数据空间，ii) 采用最佳子核编码局部区域以发现细粒度差异，以及iii) 为专业领域学习注意力机制。在三个低资源任务上的实验表明，我们的方案已提供了比迁移学习、数据增强和细粒度方法更优的基线。这凸显了低资源视觉对基础模型的独特特征和挑战，值得进一步探究。项目页面：https://xiaobai1217.github.io/Low-Resource-Vision/。