The growing demand for robust scene understanding in mobile robotics and autonomous driving has highlighted the importance of integrating multiple sensing modalities. By combining data from diverse sensors like cameras and LIDARs, fusion techniques can overcome the limitations of individual sensors, enabling a more complete and accurate perception of the environment. We introduce a novel approach to multi-modal sensor fusion, focusing on developing a graph-based state representation that supports critical decision-making processes in autonomous driving. We present a Sensor-Agnostic Graph-Aware Kalman Filter [3], the first online state estimation technique designed to fuse multi-modal graphs derived from noisy multi-sensor data. The estimated graph-based state representations serve as a foundation for advanced applications like Multi-Object Tracking (MOT), offering a comprehensive framework for enhancing the situational awareness and safety of autonomous systems. We validate the effectiveness of our proposed framework through extensive experiments conducted on both synthetic and real-world driving datasets (nuScenes). Our results showcase an improvement in MOTA and a reduction in estimated position errors (MOTP) and identity switches (IDS) for tracked objects using the SAGA-KF. Furthermore, we highlight the capability of such a framework to develop methods that can leverage heterogeneous information (like semantic objects and geometric structures) from various sensing modalities, enabling a more holistic approach to scene understanding and enhancing the safety and effectiveness of autonomous systems.

翻译：移动机器人学和自动驾驶领域对鲁棒场景理解的日益增长的需求，突显了整合多种感知模态的重要性。通过融合来自摄像头和激光雷达等不同传感器的数据，融合技术能够克服单一传感器的局限性，实现对环境更完整、更精确的感知。我们提出了一种新颖的多模态传感器融合方法，重点在于开发一种基于图的状态表示，以支持自动驾驶中的关键决策过程。我们提出了一种传感器无关的图感知卡尔曼滤波器，这是首个旨在融合来自噪声多传感器数据的多模态图的在线状态估计技术。所估计的基于图的状态表示为多目标跟踪等高级应用奠定了基础，为增强自主系统的态势感知和安全性提供了一个全面的框架。我们通过在合成和真实世界驾驶数据集上进行的广泛实验，验证了所提出框架的有效性。我们的结果表明，使用SAGA-KF后，多目标跟踪精度有所提升，同时被跟踪目标的估计位置误差和身份切换次数均有所减少。此外，我们强调了此类框架具备开发能够利用来自各种感知模态的异构信息的能力，从而为场景理解提供了一种更全面的方法，并增强了自主系统的安全性和有效性。