Vision-Language Models (VLMs) are known to inherit and amplify societal biases from their web-scale training data with Indian being particularly misrepresented. Existing fairness-aware datasets have significantly improved demographic balance across global race and gender groups, yet they continue to treat Indian as a single monolithic category. The oversimplification ignores the vast intra-national diversity across 28 states and 8 Union Territories of India and leads to representational and geographical bias. To address the limitation, we present IndicFairFace, a novel and balanced face dataset comprising 14,400 images representing geographical diversity of India. Images were sourced ethically from Wikimedia Commons and open-license web repositories and uniformly balanced across states and gender. Using IndicFairFace, we quantify intra-national geographical bias in prominent CLIP-based VLMs and reduce it using post-hoc Iterative Nullspace Projection debiasing approach. We also show that the adopted debiasing approach does not adversely impact the existing embedding space as the average drop in retrieval accuracy on benchmark datasets is less than 1.5 percent. Our work establishes IndicFairFace as the first benchmark to study geographical bias in VLMs for the Indian context.

翻译：视觉语言模型（VLMs）已知会继承并放大其网络规模训练数据中的社会偏见，其中印度群体尤其受到失实表征。现有的公平性感知数据集已在全球种族与性别群体间显著改善了人口统计平衡，但它们仍将印度视为单一同质类别。这种过度简化忽视了印度28个邦和8个中央直辖区之间的巨大国内多样性，并导致表征性与地理偏见。为应对这一局限，我们提出了IndicFairFace——一个新颖且平衡的人脸数据集，包含14,400张代表印度地理多样性的图像。图像均从维基共享资源及开放许可的网络资源库中合规获取，并在各邦与性别间均匀平衡。利用IndicFairFace，我们量化了主流基于CLIP的VLMs中的国内地理偏见，并通过后处理的迭代零空间投影去偏方法降低了该偏见。我们还证明，所采用的去偏方法不会对现有嵌入空间产生负面影响，其在基准数据集上的检索准确率平均下降幅度小于1.5%。本工作确立了IndicFairFace作为首个研究印度语境下VLMs地理偏见的基准数据集。