Despite the impressive performance of current AI models reported across various tasks, performance reports often do not include evaluations of how these models perform on the specific groups that will be impacted by these technologies. Among the minority groups under-represented in AI, data from low-income households are often overlooked in data collection and model evaluation. We evaluate the performance of a state-of-the-art vision-language model (CLIP) on a geo-diverse dataset containing household images associated with different income values (Dollar Street) and show that performance inequality exists among households of different income levels. Our results indicate that performance for the poorer groups is consistently lower than the wealthier groups across various topics and countries. We highlight insights that can help mitigate these issues and propose actionable steps for economic-level inclusive AI development. Code is available at https://github.com/MichiganNLP/Bridging_the_Digital_Divide.

翻译：尽管当前AI模型在各类任务中展现出令人瞩目的性能，但相关报告往往缺乏对这些技术所影响特定群体表现情况的评估。在AI领域代表性不足的少数群体中，低收入家庭的数据常在数据收集和模型评估中被忽视。本文基于包含不同收入水平家庭图像的地理多样性数据集（Dollar Street），对当前最先进的视觉-语言模型CLIP进行性能评估，结果表明不同收入阶层的家庭间存在性能不平等现象。我们的研究发现，在各主题和国家背景下，低收入群体的模型性能始终低于高收入群体。本文着重阐述了缓解此类问题的关键见解，并提出了推动经济包容性人工智能发展的可行措施。相关代码已开源在 https://github.com/MichiganNLP/Bridging_the_Digital_Divide。