Multi-Agent Path Finding (MAPF) aims to arrange collision-free goal-reaching paths for a group of agents. Anytime MAPF solvers based on large neighborhood search (LNS) have gained prominence recently due to their flexibility and scalability. Neighborhood selection strategy is crucial to the success of MAPF-LNS and a flurry of methods have been proposed. However, several pitfalls exist and hinder a comprehensive evaluation of these new methods, which mainly include: 1) Lower than actual or incorrect baseline performance; 2) Lack of a unified evaluation setting and criterion; 3) Lack of a codebase or executable model for supervised learning methods. To overcome these challenges, we conduct a fair comparison across prominent methods on the same benchmark and hyperparameter search settings. Additionally, we propose a simple neighborhood selection strategy which marks a clear advancement in terms of runtime efficiency in large maps with large number of agents. Our benchmarking evaluation promotes new challenges for existing learning based methods and presents opportunities for future research when machine learning is integrated with MAPF-LNS. Code and data are available at https://github.com/ChristinaTan0704/mapf-lns-benchmark.

翻译：多智能体路径规划（MAPF）旨在为一组智能体规划无碰撞的抵达目标路径。基于大邻域搜索（LNS）的任意时间MAPF求解器因其灵活性和可扩展性近年来备受关注。邻域选择策略对MAPF-LNS的成功至关重要，已有大量方法被提出。然而，现有评估存在若干缺陷，阻碍了对这些新方法的全面评估，主要包括：1）基线性能低于实际水平或存在错误；2）缺乏统一的评估设置与标准；3）监督学习方法缺少代码库或可执行模型。为应对这些挑战，我们在相同基准测试和超参数搜索设置下对主流方法进行了公平比较。此外，我们提出了一种简单的邻域选择策略，该策略在具有大量智能体的大规模地图中，于运行时间效率方面实现了显著提升。我们的基准测试评估为现有基于学习的方法提出了新挑战，并为机器学习与MAPF-LNS结合的未来研究提供了机遇。代码与数据可通过 https://github.com/ChristinaTan0704/mapf-lns-benchmark 获取。