The adoption of deep neural networks (DNNs) in safety-critical contexts is often prevented by the lack of effective means to explain their results, especially when they are erroneous. In our previous work, we proposed a white-box approach (HUDD) and a black-box approach (SAFE) to automatically characterize DNN failures. They both identify clusters of similar images from a potentially large set of images leading to DNN failures. However, the analysis pipelines for HUDD and SAFE were instantiated in specific ways according to common practices, deferring the analysis of other pipelines to future work. In this paper, we report on an empirical evaluation of 99 different pipelines for root cause analysis of DNN failures. They combine transfer learning, autoencoders, heatmaps of neuron relevance, dimensionality reduction techniques, and different clustering algorithms. Our results show that the best pipeline combines transfer learning, DBSCAN, and UMAP. It leads to clusters almost exclusively capturing images of the same failure scenario, thus facilitating root cause analysis. Further, it generates distinct clusters for each root cause of failure, thus enabling engineers to detect all the unsafe scenarios. Interestingly, these results hold even for failure scenarios that are only observed in a small percentage of the failing images.

翻译：深度神经网络在安全关键场景中的应用常因其结果缺乏有效解释手段（尤其在出现错误时）而受阻。我们先前研究提出了白盒方法（HUDD）与黑盒方法（SAFE）以自动表征DNN失效模式，两者均能从潜在大量图像中识别出导致DNN失效的相似图像集群。然而，HUDD与SAFE的分析流程仅按常规实践以特定方式实现，其余分析流程有待后续研究。本文对99种用于DNN失效根因分析的不同流程进行了实证评估。这些流程融合了迁移学习、自编码器、神经元相关热力图、降维技术及不同聚类算法。结果表明，最优流程整合了迁移学习、DBSCAN与UMAP。该流程生成的聚类几乎仅包含相同失效场景的图像，从而简化根因分析；同时为每种失效根因生成独立聚类，使工程师能够检测所有不安全场景。值得注意的是，即使对于仅存在于极小比例失效图像中的场景，上述结论依然成立。