Trading off performance guarantees in favor of scalability, the Multi-Agent Path Finding (MAPF) community has recently started to embrace Multi-Agent Reinforcement Learning (MARL), where agents learn to collaboratively generate individual, collision-free (but often suboptimal) paths. Scalability is usually achieved by assuming a local field of view (FOV) around the agents, helping scale to arbitrary world sizes. However, this assumption significantly limits the amount of information available to the agents, making it difficult for them to enact the type of joint maneuvers needed in denser MAPF tasks. In this paper, we propose SCRIMP, where agents learn individual policies from even very small (down to 3x3) FOVs, by relying on a highly-scalable global/local communication mechanism based on a modified transformer. We further equip agents with a state-value-based tie-breaking strategy to further improve performance in symmetric situations, and introduce intrinsic rewards to encourage exploration while mitigating the long-term credit assignment problem. Empirical evaluations on a set of experiments indicate that SCRIMP can achieve higher performance with improved scalability compared to other state-of-the-art learning-based MAPF planners with larger FOVs, and even yields similar performance as a classical centralized planner in many cases. Ablation studies further validate the effectiveness of our proposed techniques. Finally, we show that our trained model can be directly implemented on real robots for online MAPF through high-fidelity simulations in gazebo.

翻译：多智能体路径规划（MAPF）领域为兼顾可扩展性而牺牲性能保证，近年来开始采用多智能体强化学习（MARL）方法，使智能体通过协作生成各自无碰撞（但通常非最优）的路径。可扩展性通常通过假设智能体周围存在局部视野（FOV）实现，这有助于扩展至任意规模的世界。然而，这一假设严重限制了智能体可获取的信息量，使其难以在密集MAPF任务中执行所需的联合机动操作。本文提出SCRIMP方法，该方法基于改进型Transformer构建高度可扩展的全局/局部通信机制，使智能体即使在极小（小至3×3）视野下也能学习独立策略。我们进一步为智能体配备基于状态值的破平策略以提升对称场景下的性能，并引入内在奖励机制以激励探索，同时缓解长期信用分配问题。实验评估表明，相比其他采用更大视野的先进学习型MAPF规划器，SCRIMP能以更优可扩展性实现更高性能，甚至在许多情况下达到经典集中式规划器的相似水平。消融研究进一步验证了所提技术的有效性。最后，通过Gazebo高保真仿真证明，我们的训练模型可直接部署于真实机器人，实现在线MAPF。