Ensuring that all classes of objects are detected with equal accuracy is essential in AI systems. For instance, being unable to identify any one class of objects could have fatal consequences in autonomous driving systems. Hence, ensuring the reliability of image recognition systems is crucial. This work addresses how to validate group fairness in image recognition software. We propose a distribution-aware fairness testing approach (called DistroFair) that systematically exposes class-level fairness violations in image classifiers via a synergistic combination of out-of-distribution (OOD) testing and semantic-preserving image mutation. DistroFair automatically learns the distribution (e.g., number/orientation) of objects in a set of images. Then it systematically mutates objects in the images to become OOD using three semantic-preserving image mutations - object deletion, object insertion and object rotation. We evaluate DistroFair using two well-known datasets (CityScapes and MS-COCO) and three major, commercial image recognition software (namely, Amazon Rekognition, Google Cloud Vision and Azure Computer Vision). Results show that about 21% of images generated by DistroFair reveal class-level fairness violations using either ground truth or metamorphic oracles. DistroFair is up to 2.3x more effective than two main baselines, i.e., (a) an approach which focuses on generating images only within the distribution (ID) and (b) fairness analysis using only the original image dataset. We further observed that DistroFair is efficient, it generates 460 images per hour, on average. Finally, we evaluate the semantic validity of our approach via a user study with 81 participants, using 30 real images and 30 corresponding mutated images generated by DistroFair. We found that images generated by DistroFair are 80% as realistic as real-world images.

翻译：确保所有类别对象被等精度检测是AI系统的关键要求。例如，在自动驾驶系统中，无法识别任一类别物体可能导致致命后果。因此，确保图像识别系统的可靠性至关重要。本文旨在解决图像识别软件中群体公平性的验证问题。我们提出了一种分布感知的公平性测试方法（DistroFair），该方法通过离群分布（OOD）测试与语义保持图像变异的协同组合，系统性地暴露图像分类器中的类别级公平性违规。DistroFair自动学习图像集合中对象的分布（如数量/朝向），进而通过三种语义保持的图像变异——对象删除、对象插入和对象旋转——将图像中的对象变异为离群分布。我们使用两个知名数据集（CityScapes和MS-COCO）以及三款主流商用图像识别软件（Amazon Rekognition、Google Cloud Vision和Azure Computer Vision）对DistroFair进行评估。结果表明，DistroFair生成的图像中约21%通过基准真值或蜕变预言机暴露了类别级公平性违规。DistroFair的有效性相比两种主要基线方法提升达2.3倍：（a）仅生成分布内（ID）图像的方法，（b）仅使用原始图像数据集进行公平性分析。我们还观察到DistroFair具有高效性，平均每小时生成460张图像。最后，我们通过一项包含81名参与者的用户研究评估方法的语义有效性，采用30张真实图像及DistroFair生成的30张对应变异图像。研究发现，DistroFair生成的图像真实度达到真实世界图像的80%。