In Autonomous Driving (AD) transparency and safety are paramount, as mistakes are costly. However, neural networks used in AD systems are generally considered black boxes. As a countermeasure, we have methods of explainable AI (XAI), such as feature relevance estimation and dimensionality reduction. Coarse graining techniques can also help reduce dimensionality and find interpretable global patterns. A specific coarse graining method is Renormalization Groups from statistical physics. It has previously been applied to Restricted Boltzmann Machines (RBMs) to interpret unsupervised learning. We refine this technique by building a transparent backbone model for convolutional variational autoencoders (VAE) that allows mapping latent values to input features and has performance comparable to trained black box VAEs. Moreover, we propose a custom feature map visualization technique to analyze the internal convolutional layers in the VAE to explain internal causes of poor reconstruction that may lead to dangerous traffic scenarios in AD applications. In a second key contribution, we propose explanation and evaluation techniques for the internal dynamics and feature relevance of prediction networks. We test a long short-term memory (LSTM) network in the computer vision domain to evaluate the predictability and in future applications potentially safety of prediction models. We showcase our methods by analyzing a VAE-LSTM world model that predicts pedestrian perception in an urban traffic situation.

翻译：在自动驾驶（AD）中，透明性与安全性至关重要，因为错误代价高昂。然而，自动驾驶系统中使用的神经网络通常被视为黑箱。作为应对，我们采用可解释人工智能（XAI）方法，如特征相关性估计与降维技术。粗粒化技术同样有助于降低维度并识别可解释的全局模式。其中一种特定的粗粒化方法源自统计物理中的重整化群，此前已被应用于受限玻尔兹曼机（RBM）以解释无监督学习。我们对该技术进行改进，构建了一个针对卷积变分自编码器（VAE）的透明骨干模型——该模型既能将潜在变量映射到输入特征，又具备与经过训练的黑箱VAE相当的性能。此外，我们还提出一种自定义特征图可视化技术，用于分析VAE内部卷积层，解释可能导致AD危险交通场景的低质量重建的内在原因。在第二项关键贡献中，我们提出了预测网络内部动力学与特征相关性的解释与评估技术。我们通过在计算机视觉领域测试长短期记忆（LSTM）网络，评估预测模型的可预测性，并在未来应用中评估其潜在安全性。最后，我们通过分析一个预测城市交通场景中行人感知的VAE-LSTM世界模型来展示所提出的方法。