We propose SLAMFuse, an open-source SLAM benchmarking framework that provides consistent crossplatform environments for evaluating multi-modal SLAM algorithms, along with tools for data fuzzing, failure detection, and diagnosis across different datasets. Our framework introduces a fuzzing mechanism to test the resilience of SLAM algorithms against dataset perturbations. This enables the assessment of pose estimation accuracy under varying conditions and identifies critical perturbation thresholds. SLAMFuse improves diagnostics with failure detection and analysis tools, examining algorithm behaviour against dataset characteristics. SLAMFuse uses Docker to ensure reproducible testing conditions across diverse datasets and systems by streamlining dependency management. Emphasizing the importance of reproducibility and introducing advanced tools for algorithm evaluation and performance diagnosis, our work sets a new precedent for reliable benchmarking of SLAM systems. We provide ready-to-use docker compatible versions of the algorithms and datasets used in the experiments, together with guidelines for integrating and benchmarking new algorithms. Code is available at https://github.com/nikolaradulov/slamfuse

翻译：我们提出了SLAMFuse，这是一个开源的SLAM基准测试框架，为评估多模态SLAM算法提供一致的跨平台环境，同时提供数据模糊测试、故障检测以及跨不同数据集的诊断工具。该框架引入模糊测试机制，以测试SLAM算法对数据集扰动的鲁棒性。这使得能够在不同条件下评估位姿估计精度，并识别关键的扰动阈值。SLAMFuse通过故障检测与分析工具改进了诊断能力，根据数据集特征检验算法行为。SLAMFuse利用Docker简化依赖管理，确保在不同数据集和系统间实现可复现的测试条件。我们的工作强调了可复现性的重要性，并引入了用于算法评估与性能诊断的先进工具，为SLAM系统的可靠基准测试设立了新标准。我们提供了实验中使用的算法和数据集的即用型Docker兼容版本，以及集成与基准测试新算法的指南。代码发布于https://github.com/nikolaradulov/slamfuse。