Object detection is a fundamental task in computer vision, which has been greatly progressed through developing large and intricate deep learning models. However, the lack of transparency is a big challenge that may not allow the widespread adoption of these models. Explainable artificial intelligence is a field of research where methods are developed to help users understand the behavior, decision logics, and vulnerabilities of AI-based systems. Black-box explanation refers to explaining decisions of an AI system without having access to its internals. In this paper, we design and implement a black-box explanation method named Black-box Object Detection Explanation by Masking (BODEM) through adopting a new masking approach for AI-based object detection systems. We propose local and distant masking to generate multiple versions of an input image. Local masks are used to disturb pixels within a target object to figure out how the object detector reacts to these changes, while distant masks are used to assess how the detection model's decisions are affected by disturbing pixels outside the object. A saliency map is then created by estimating the importance of pixels through measuring the difference between the detection output before and after masking. Finally, a heatmap is created that visualizes how important pixels within the input image are to the detected objects. The experimentations on various object detection datasets and models showed that BODEM can be effectively used to explain the behavior of object detectors and reveal their vulnerabilities. This makes BODEM suitable for explaining and validating AI based object detection systems in black-box software testing scenarios. Furthermore, we conducted data augmentation experiments that showed local masks produced by BODEM can be used for further training the object detectors and improve their detection accuracy and robustness.

翻译：摘要：目标检测是计算机视觉中的一项基础任务，近年来通过构建大型且复杂的深度学习模型取得了显著进展。然而，透明度的缺乏是阻碍这些模型广泛应用的重大挑战。可解释人工智能是一个研究领域，其旨在开发方法帮助用户理解基于AI的系统的行为、决策逻辑及脆弱性。黑盒解释是指在不访问AI系统内部结构的情况下解释其决策。本文设计并实现了一种名为“通过掩码实现黑盒目标检测解释”（BODEM）的黑盒解释方法，该方法采用新的掩码技术用于基于AI的目标检测系统。我们提出了局部掩码和远距离掩码来生成输入图像的多个版本。局部掩码用于干扰目标对象内的像素，以探究目标检测器对这些变化的响应；而远距离掩码则通过干扰对象外的像素，评估检测模型决策如何受到影响。随后，通过计算掩码前后检测输出的差异来估计像素重要性，从而生成显著性图。最终创建热力图，可视化输入图像中哪些像素对检测到的目标至关重要。在多个目标检测数据集和模型上的实验表明，BODEM能够有效解释目标检测器的行为并揭示其脆弱性，使其适用于黑盒软件测试场景中对基于AI的目标检测系统进行解释和验证。此外，我们进行了数据增强实验，结果显示BODEM生成的局部掩码可用于进一步训练目标检测器，提高其检测精度和鲁棒性。