The adoption of deep neural networks (DNNs) in safety-critical contexts is often prevented by the lack of effective means to explain their results, especially when they are erroneous. In our previous work, we proposed a white-box approach (HUDD) and a black-box approach (SAFE) to automatically characterize DNN failures. They both identify clusters of similar images from a potentially large set of images leading to DNN failures. However, the analysis pipelines for HUDD and SAFE were instantiated in specific ways according to common practices, deferring the analysis of other pipelines to future work. In this paper, we report on an empirical evaluation of 99 different pipelines for root cause analysis of DNN failures. They combine transfer learning, autoencoders, heatmaps of neuron relevance, dimensionality reduction techniques, and different clustering algorithms. Our results show that the best pipeline combines transfer learning, DBSCAN, and UMAP. It leads to clusters almost exclusively capturing images of the same failure scenario, thus facilitating root cause analysis. Further, it generates distinct clusters for each root cause of failure, thus enabling engineers to detect all the unsafe scenarios. Interestingly, these results hold even for failure scenarios that are only observed in a small percentage of the failing images.

翻译：深度神经网络在安全关键领域的应用常因缺乏有效解释其结果的机制（尤其是结果错误时）而受阻。我们之前的工作提出了白盒方法HUDD与黑盒方法SAFE，用于自动表征DNN失效模式。这两种方法都能从可能导致DNN失效的潜在大量图像中识别出相似图像聚类。然而，HUDD与SAFE的分析流程依据常规实践以特定方式实例化，将其他流程的分析留待后续研究。本文报告了对99种不同DNN失效根因分析流程的实证评估，这些流程融合了迁移学习、自编码器、神经元相关性热图、降维技术及多种聚类算法。结果表明，最优流程结合了迁移学习、DBSCAN和UMAP，生成的聚类几乎仅包含相同失效场景的图像，从而简化根因分析。此外，该流程能为每种失效根因生成独立聚类，使工程师能够检测所有不安全场景。值得注意的是，即使对于仅在极小比例失效图像中出现的失效场景，该结果依然成立。