For assistive robots, one critical use case of SLAM is to support localization as they navigate through an environment completing tasks. Current SLAM benchmarks do not consider task-based deployments where repeatability (precision) is more critical than accuracy. To address this gap, we propose a task-driven benchmarking framework for evaluating SLAM methods. The framework accounts for SLAM's mapping capabilities, employs precision as a key metric, and has low resource requirements to implement. Testing of state-of-the-art SLAM methods in both simulated and real-world scenarios provides insights into the performance properties of modern SLAM solutions. In particular, it shows that passive stereo SLAM operates at a level of precision comparable to LiDAR-based SLAM in typical indoor environments. The benchmarking approach offers a more relevant and accurate assessment of SLAM performance in task-driven applications.

翻译：对于辅助机器人而言，SLAM的一个关键应用场景是在其执行任务导航过程中提供定位支持。现有SLAM基准测试未考虑任务型部署场景，此类场景中可重复性（精度）比绝对精度更为关键。为填补这一空白，我们提出了一种用于评估SLAM方法的任务驱动基准测试框架。该框架综合考虑SLAM的建图能力，采用精度作为核心评价指标，且实现所需的资源成本较低。通过在仿真和真实场景中对前沿SLAM方法进行测试，本研究揭示了现代SLAM解决方案的性能特性。特别值得注意的是，实验表明在典型室内环境中，被动立体视觉SLAM能达到与基于激光雷达的SLAM相当的精度水平。该基准测试方法为任务驱动应用中的SLAM性能评估提供了更具相关性和准确性的衡量标准。