Multi-Sensor Fusion (MSF) based perception systems have been the foundation in supporting many industrial applications and domains, such as self-driving cars, robotic arms, and unmanned aerial vehicles. Over the past few years, the fast progress in data-driven artificial intelligence (AI) has brought a fast-increasing trend to empower MSF systems by deep learning techniques to further improve performance, especially on intelligent systems and their perception systems. Although quite a few AI-enabled MSF perception systems and techniques have been proposed, up to the present, limited benchmarks that focus on MSF perception are publicly available. Given that many intelligent systems such as self-driving cars are operated in safety-critical contexts where perception systems play an important role, there comes an urgent need for a more in-depth understanding of the performance and reliability of these MSF systems. To bridge this gap, we initiate an early step in this direction and construct a public benchmark of AI-enabled MSF-based perception systems including three commonly adopted tasks (i.e., object detection, object tracking, and depth completion). Based on this, to comprehensively understand MSF systems' robustness and reliability, we design 14 common and realistic corruption patterns to synthesize large-scale corrupted datasets. We further perform a systematic evaluation of these systems through our large-scale evaluation. Our results reveal the vulnerability of the current AI-enabled MSF perception systems, calling for researchers and practitioners to take robustness and reliability into account when designing AI-enabled MSF.

翻译：基于多传感器融合的感知系统已成为支持自动驾驶汽车、机械臂和无人机等众多工业应用与领域的基石。近年来，数据驱动人工智能的快速发展催生了利用深度学习技术赋能多传感器融合系统以进一步提升性能（尤其是智能系统及其感知系统性能）的迅猛趋势。尽管已有多种AI赋能的多传感器融合感知系统和技术被提出，但迄今为止，公开可用的聚焦于多传感器融合感知的基准测试仍然有限。鉴于自动驾驶汽车等众多智能系统运行在安全关键场景中，且感知系统在其中扮演重要角色，亟需更深入地理解这些多传感器融合系统的性能与可靠性。为填补这一空白，我们在此方向上迈出初步探索，构建了包含三项常见任务（即目标检测、目标跟踪和深度补全）的AI赋能多传感器融合感知系统公开基准。基于此，为全面理解多传感器融合系统的鲁棒性与可靠性，我们设计了14种常见且逼真的损坏模式以生成大规模损坏数据集。通过系统化的大规模评估，我们的结果揭示了当前AI赋能多传感器融合感知系统的脆弱性，呼吁研究人员与从业者在设计AI赋能的多传感器融合系统时将鲁棒性与可靠性纳入考量。