Trading off performance guarantees in favor of scalability, the Multi-Agent Path Finding (MAPF) community has recently started to embrace Multi-Agent Reinforcement Learning (MARL), where agents learn to collaboratively generate individual, collision-free (but often suboptimal) paths. Scalability is usually achieved by assuming a local field of view (FOV) around the agents, helping scale to arbitrary world sizes. However, this assumption significantly limits the amount of information available to the agents, making it difficult for them to enact the type of joint maneuvers needed in denser MAPF tasks. In this paper, we propose SCRIMP, where agents learn individual policies from even very small (down to 3x3) FOVs, by relying on a highly-scalable global/local communication mechanism based on a modified transformer. We further equip agents with a state-value-based tie-breaking strategy to further improve performance in symmetric situations, and introduce intrinsic rewards to encourage exploration while mitigating the long-term credit assignment problem. Empirical evaluations on a set of experiments indicate that SCRIMP can achieve higher performance with improved scalability compared to other state-of-the-art learning-based MAPF planners with larger FOVs, and even yields similar performance as a classical centralized planner in many cases. Ablation studies further validate the effectiveness of our proposed techniques. Finally, we show that our trained model can be directly implemented on real robots for online MAPF through high-fidelity simulations in gazebo.

翻译：多智能体路径规划（MAPF）领域的研究人员近期开始采用多智能体强化学习（MARL），以牺牲性能保证为代价换取可扩展性，在该框架下智能体通过协作生成各自无碰撞（但通常次优）的路径。可扩展性通常通过假设智能体具备局部视野（FOV）实现，这有助于将方法推广至任意规模的世界。然而，这种假设严重限制了智能体可获取的信息量，使其难以在更密集的MAPF任务中执行所需的联合协同操作。本文提出SCRIMP框架，智能体通过基于改进Transformer的高度可扩展全局/局部通信机制，即使利用极小（小至3×3）的视野也能学习独立策略。我们进一步为智能体配备基于状态价值的破平策略以提升对称场景下的性能，并引入内在奖励以鼓励探索，同时缓解长期信用分配问题。基于系列实验的实证评估表明，与采用更大视野的其他前沿学习型MAPF规划器相比，SCRIMP在实现更高可扩展性的同时取得了更优性能，并在多数情况下甚至达到与传统集中式规划器相当的水平。消融研究进一步验证了所提技术的有效性。最后，我们通过Gazebo高保真仿真证明，训练后的模型可直接部署于真实机器人以实现在线MAPF。