Data augmentation has become a pivotal tool in enhancing the performance of computer vision tasks, with the KeepOriginalAugment method emerging as a standout technique for its intelligent incorporation of salient regions within less prominent areas, enabling augmentation in both regions. Despite its success in image classification, its potential in addressing biases remains unexplored. In this study, we introduce an extension of the KeepOriginalAugment method, termed FaceKeepOriginalAugment, which explores various debiasing aspects-geographical, gender, and stereotypical biases-in computer vision models. By maintaining a delicate balance between data diversity and information preservation, our approach empowers models to exploit both diverse salient and non-salient regions, thereby fostering increased diversity and debiasing effects. We investigate multiple strategies for determining the placement of the salient region and swapping perspectives to decide which part undergoes augmentation. Leveraging the Image Similarity Score (ISS), we quantify dataset diversity across a range of datasets, including Flickr Faces HQ (FFHQ), WIKI, IMDB, Labelled Faces in the Wild (LFW), UTK Faces, and Diverse Dataset. We evaluate the effectiveness of FaceKeepOriginalAugment in mitigating gender bias across CEO, Engineer, Nurse, and School Teacher datasets, utilizing the Image-Image Association Score (IIAS) in convolutional neural networks (CNNs) and vision transformers (ViTs). Our findings shows the efficacy of FaceKeepOriginalAugment in promoting fairness and inclusivity within computer vision models, demonstrated by reduced gender bias and enhanced overall fairness. Additionally, we introduce a novel metric, Saliency-Based Diversity and Fairness Metric, which quantifies both diversity and fairness while handling data imbalance across various datasets.

翻译：数据增强已成为提升计算机视觉任务性能的关键工具，其中KeepOriginalAugment方法因其能智能地将显著区域融入非显著区域、实现双重区域增强而成为突出技术。尽管该方法在图像分类中取得成功，但其在解决偏见方面的潜力尚未被探索。本研究提出了KeepOriginalAugment方法的扩展版本——FaceKeepOriginalAugment，旨在探究计算机视觉模型中地理、性别和刻板印象等多维度去偏问题。通过在数据多样性与信息保留间保持精细平衡，我们的方法使模型能够同时利用多样化的显著与非显著区域，从而增强多样性并产生去偏效果。我们研究了多种确定显著区域放置位置的策略，并通过交换视角决定哪些部分进行增强。借助图像相似度评分（ISS），我们对包括Flickr Faces HQ（FFHQ）、WIKI、IMDB、野外标记人脸（LFW）、UTK人脸和多样化数据集在内的多个数据集进行了多样性量化。通过卷积神经网络（CNN）和视觉Transformer（ViT）中的图像-图像关联评分（IIAS），我们评估了FaceKeepOriginalAugment在CEO、工程师、护士和学校教师数据集中缓解性别偏见的效果。研究结果表明，FaceKeepOriginalAugment能有效促进计算机视觉模型的公平性与包容性，具体表现为性别偏见减少和整体公平性提升。此外，我们提出了一种新颖的度量标准——基于显著性的多样性与公平性度量，该标准在量化多样性与公平性的同时，能有效处理不同数据集间的数据不平衡问题。