The adoption of deep neural networks (DNNs) in safety-critical contexts is often prevented by the lack of effective means to explain their results, especially when they are erroneous. In our previous work, we proposed a white-box approach (HUDD) and a black-box approach (SAFE) to automatically characterize DNN failures. They both identify clusters of similar images from a potentially large set of images leading to DNN failures. However, the analysis pipelines for HUDD and SAFE were instantiated in specific ways according to common practices, deferring the analysis of other pipelines to future work. In this paper, we report on an empirical evaluation of 99 different pipelines for root cause analysis of DNN failures. They combine transfer learning, autoencoders, heatmaps of neuron relevance, dimensionality reduction techniques, and different clustering algorithms. Our results show that the best pipeline combines transfer learning, DBSCAN, and UMAP. It leads to clusters almost exclusively capturing images of the same failure scenario, thus facilitating root cause analysis. Further, it generates distinct clusters for each root cause of failure, thus enabling engineers to detect all the unsafe scenarios. Interestingly, these results hold even for failure scenarios that are only observed in a small percentage of the failing images.

翻译：在安全关键场景中，深度神经网络（DNN）的应用常因缺乏有效手段解释其输出结果（尤其是错误输出）而受阻。在先前工作中，我们分别提出了白盒方法（HUDD）与黑盒方法（SAFE）来自动表征DNN故障。这两种方法均能从可能引发DNN故障的大规模图像集中识别出相似图像簇。然而，HUDD与SAFE的分析流程是根据常规实践以特定方式实例化的，其他分析流程的分析工作留待未来研究。本文对99种用于DNN故障根因分析的不同流程进行了实证评估。这些流程融合了迁移学习、自编码器、神经元相关性热力图、降维技术及不同聚类算法。结果表明，最优流程结合了迁移学习、DBSCAN与UMAP。该流程生成的簇几乎仅包含记录同一故障场景的图像，从而显著简化了根因分析。此外，该流程为每种故障根因生成独立簇，使工程师能够检测所有不安全场景。值得注意的是，即使故障场景仅出现在极小比例（如少数百分比）的故障图像中，上述结论依然成立。