Ensuring that all classes of objects are detected with equal accuracy is essential in AI systems. For instance, being unable to identify any one class of objects could have fatal consequences in autonomous driving systems. Hence, ensuring the reliability of image recognition systems is crucial. This work addresses how to validate group fairness in image recognition software. We propose a distribution-aware fairness testing approach (called DistroFair) that systematically exposes class-level fairness violations in image classifiers via a synergistic combination of out-of-distribution (OOD) testing and semantic-preserving image mutation. DistroFair automatically learns the distribution (e.g., number/orientation) of objects in a set of images. Then it systematically mutates objects in the images to become OOD using three semantic-preserving image mutations - object deletion, object insertion and object rotation. We evaluate DistroFair using two well-known datasets (CityScapes and MS-COCO) and three major, commercial image recognition software (namely, Amazon Rekognition, Google Cloud Vision and Azure Computer Vision). Results show that about 21% of images generated by DistroFair reveal class-level fairness violations using either ground truth or metamorphic oracles. DistroFair is up to 2.3x more effective than two main baselines, i.e., (a) an approach which focuses on generating images only within the distribution (ID) and (b) fairness analysis using only the original image dataset. We further observed that DistroFair is efficient, it generates 460 images per hour, on average. Finally, we evaluate the semantic validity of our approach via a user study with 81 participants, using 30 real images and 30 corresponding mutated images generated by DistroFair. We found that images generated by DistroFair are 80% as realistic as real-world images.

翻译：确保各类对象被同等准确地检测是人工智能系统的关键要求。例如，在自动驾驶系统中无法识别任何一类对象可能导致致命后果。因此，确保图像识别系统的可靠性至关重要。本文旨在解决图像识别软件中群体公平性的验证问题。我们提出一种分布感知的公平性测试方法（称为DistroFair），通过结合分布外测试与语义保持图像变异，系统性地暴露图像分类器中类级公平性违规。DistroFair自动学习图像集合中对象的分布（如数量/方向），然后通过三种语义保持的图像变异（对象删除、对象插入和对象旋转）系统性地将图像中的对象变异为分布外样本。我们使用两个知名数据集（CityScapes和MS-COCO）以及三个主流商业图像识别软件（即Amazon Rekognition、Google Cloud Vision和Azure Computer Vision）对DistroFair进行评估。结果表明，通过真实标注或变形神谕验证，DistroFair生成的图像中约有21%揭示了类级公平性违规。DistroFair的效果比两个主要基线方法高出2.3倍：（a）仅关注生成分布内图像的方法，（b）仅使用原始图像数据集进行公平性分析的方法。我们还观察到DistroFair效率较高，平均每小时生成460张图像。最后，我们通过一项包含81名参与者的用户研究，使用30张真实图像和30张由DistroFair生成的对应变异图像，评估了方法的语义有效性。研究发现，DistroFair生成的图像真实度达到真实世界图像的80%。