Trading off performance guarantees in favor of scalability, the Multi-Agent Path Finding (MAPF) community has recently started to embrace Multi-Agent Reinforcement Learning (MARL), where agents learn to collaboratively generate individual, collision-free (but often suboptimal) paths. Scalability is usually achieved by assuming a local field of view (FOV) around the agents, helping scale to arbitrary world sizes. However, this assumption significantly limits the amount of information available to the agents, making it difficult for them to enact the type of joint maneuvers needed in denser MAPF tasks. In this paper, we propose SCRIMP, where agents learn individual policies from even very small (down to 3x3) FOVs, by relying on a highly-scalable global/local communication mechanism based on a modified transformer. We further equip agents with a state-value-based tie-breaking strategy to further improve performance in symmetric situations, and introduce intrinsic rewards to encourage exploration while mitigating the long-term credit assignment problem. Empirical evaluations on a set of experiments indicate that SCRIMP can achieve higher performance with improved scalability compared to other state-of-the-art learning-based MAPF planners with larger FOVs, and even yields similar performance as a classical centralized planner in many cases. Ablation studies further validate the effectiveness of our proposed techniques. Finally, we show that our trained model can be directly implemented on real robots for online MAPF through high-fidelity simulations in gazebo.

翻译：为换取可扩展性而牺牲性能保证，多智能体路径规划（MAPF）领域近来开始采用多智能体强化学习（MARL），其中智能体通过协作学习生成各自无碰撞（但通常非最优）的路径。可扩展性通常通过假设智能体周围具有局部视野（FOV）来实现，这有助于扩展到任意大小的世界。然而，这一假设严重限制了智能体可获取的信息量，使其难以在更密集的MAPF任务中实施所需的联合机动操作。在本文中，我们提出SCRIMP框架，该框架通过基于改进型Transformer的高度可扩展的全局/局部通信机制，使智能体能够从极小的（小至3×3）视野中学习个体策略。我们进一步为智能体配备了基于状态价值的破平策略，以提升对称情境下的性能，并引入内在奖励以鼓励探索，同时缓解长期信用分配问题。一系列实验的实证评估表明，与采用更大视野的先进基于学习的MAPF规划器相比，SCRIMP能以更优的可扩展性实现更高性能，甚至在许多情况下能达到与经典集中式规划器相当的性能。消融研究进一步验证了我们所提出技术的有效性。最后，我们通过Gazebo中的高保真仿真展示了训练后的模型可直接部署于真实机器人，实现在线MAPF。