Multi-Sensor Fusion (MSF) based perception systems have been the foundation in supporting many industrial applications and domains, such as self-driving cars, robotic arms, and unmanned aerial vehicles. Over the past few years, the fast progress in data-driven artificial intelligence (AI) has brought a fast-increasing trend to empower MSF systems by deep learning techniques to further improve performance, especially on intelligent systems and their perception systems. Although quite a few AI-enabled MSF perception systems and techniques have been proposed, up to the present, limited benchmarks that focus on MSF perception are publicly available. Given that many intelligent systems such as self-driving cars are operated in safety-critical contexts where perception systems play an important role, there comes an urgent need for a more in-depth understanding of the performance and reliability of these MSF systems. To bridge this gap, we initiate an early step in this direction and construct a public benchmark of AI-enabled MSF-based perception systems including three commonly adopted tasks (i.e., object detection, object tracking, and depth completion). Based on this, to comprehensively understand MSF systems' robustness and reliability, we design 14 common and realistic corruption patterns to synthesize large-scale corrupted datasets. We further perform a systematic evaluation of these systems through our large-scale evaluation. Our results reveal the vulnerability of the current AI-enabled MSF perception systems, calling for researchers and practitioners to take robustness and reliability into account when designing AI-enabled MSF.

翻译：基于多传感器融合的感知系统已成为支撑自动驾驶汽车、机械臂及无人机等诸多工业应用与领域的基础。近年来，数据驱动的人工智能技术快速发展，推动深度学习技术赋能多传感器融合系统以进一步提升性能的趋势日益显著，尤其在智能系统及其感知领域。尽管已有多个AI赋能的多传感器融合感知系统与技术被提出，但截至目前，面向多传感器融合感知的公开基准测试仍然有限。鉴于自动驾驶汽车等众多智能系统运行于安全关键场景，感知系统在其中扮演着重要角色，亟需更深入地理解这些多传感器融合系统的性能与可靠性。为弥合这一差距，我们在此方向迈出初步探索，构建了一个面向AI赋能多传感器融合感知系统的公开基准测试，涵盖三项常见任务（即目标检测、目标追踪与深度补全）。基于此，为全面理解多传感器融合系统的鲁棒性与可靠性，我们设计了14种常见且真实的损坏模式以合成大规模损坏数据集，并通过大规模评估对这些系统进行系统性测试。研究结果揭示了当前AI赋能多传感器融合感知系统的脆弱性，呼吁研究人员与从业者在设计AI赋能多传感器融合系统时充分考虑鲁棒性与可靠性。