We use the Quality Diversity (QD) algorithm with Neural Cellular Automata (NCA) to generate benchmark maps for Multi-Agent Path Finding (MAPF) algorithms. Previously, MAPF algorithms are tested using fixed, human-designed benchmark maps. However, such fixed benchmark maps have several problems. First, these maps may not cover all the potential failure scenarios for the algorithms. Second, when comparing different algorithms, fixed benchmark maps may introduce bias leading to unfair comparisons between algorithms. In this work, we take advantage of the QD algorithm and NCA with different objectives and diversity measures to generate maps with patterns to comprehensively understand the performance of MAPF algorithms and be able to make fair comparisons between two MAPF algorithms to provide further information on the selection between two algorithms. Empirically, we employ this technique to generate diverse benchmark maps to evaluate and compare the behavior of different types of MAPF algorithms such as bounded-suboptimal algorithms, suboptimal algorithms, and reinforcement-learning-based algorithms. Through both single-planner experiments and comparisons between algorithms, we identify patterns where each algorithm excels and detect disparities in runtime or success rates between different algorithms.

翻译：本研究采用质量多样性算法结合神经细胞自动机，为多智能体路径规划算法生成基准测试地图。传统上，MAPF算法通常使用固定的人工设计基准地图进行测试，但此类固定基准地图存在若干问题：首先，这些地图可能无法覆盖算法所有潜在的失效场景；其次，在比较不同算法时，固定基准地图可能引入偏差，导致算法间的不公平比较。本研究通过配置不同目标函数与多样性度量的QD算法和NCA，生成具有特定模式的地图，以全面理解MAPF算法的性能表现，并实现两种MAPF算法间的公平比较，从而为算法选择提供更充分的决策依据。在实证研究中，我们运用该技术生成多样化基准地图，评估并比较了有界次优算法、次优算法及基于强化学习的算法等不同类型MAPF算法的行为特征。通过单规划器实验与算法间对比实验，我们识别出各类算法的优势场景，并检测到不同算法在运行时间与成功率方面存在的差异。