Deep Neural Networks have become the dominant solution for Autonomous Driving perception, but their opacity conflicts with emerging Trustworthy AI guidelines and complicates safety assurance, debugging, and human oversight. While theoretical frameworks for safe and Explainable AI (XAI) exist, concrete implementations of Trustworthy AI for 3D scene understanding remain scarce. We address this gap by proposing a Trustworthy AI perception module that is remarkably robust, integrates faithful explainability, and calibrated uncertainty estimates. Building on a transformer-based detector, we derive explanation from the attention mechanism at inference time and validate their faithfulness using perturbation-based consistency tests. We further integrate an uncertainty estimation and calibration module, and apply robustness-enhancing training methods. Experiments show faithful saliency behavior, improved robustness, and well-calibrated uncertainty estimates. Finally, we deploy these Trustworthy AI elements in a prototype vehicle and provide an XAI Interface that visualizes documentation artifacts, model uncertainty state, and saliency maps, demonstrating the feasibility of trustworthy perception monitoring in real time. Supplementary materials are available at https://tillbeemelmanns.github.io/trustworthy_ai/ .
翻译:深度神经网络已成为自动驾驶感知的主流解决方案,但其不透明性与新兴的可信人工智能指南相冲突,并增加了安全保障、调试和人工监督的复杂性。尽管针对安全与可解释人工智能(XAI)的理论框架已存在,但针对三维场景理解的可信人工智能具体实现仍然稀缺。我们通过提出一个可信人工智能感知模块来填补这一空白,该模块具有显著鲁棒性、集成忠实可解释性以及校准的不确定性估计。基于Transformer检测器,我们在推理时从注意力机制中提取解释,并使用基于扰动的连续性测试验证其忠实性。我们进一步集成不确定性估计与校准模块,并应用增强鲁棒性的训练方法。实验展示了忠实的显著性行为、改进的鲁棒性以及良好校准的不确定性估计。最后,我们将这些可信人工智能要素部署到原型车辆中,并提供可解释人工智能界面,可视化文档产物、模型不确定性状态和显著性图,从而验证实时可信感知监控的可行性。补充材料见 https://tillbeemelmanns.github.io/trustworthy_ai/。