Multi-robot path finding in dynamic environments is a highly challenging classic problem. In the movement process, robots need to avoid collisions with other moving robots while minimizing their travel distance. Previous methods for this problem either continuously replan paths using heuristic search methods to avoid conflicts or choose appropriate collision avoidance strategies based on learning approaches. The former may result in long travel distances due to frequent replanning, while the latter may have low learning efficiency due to low sample exploration and utilization, and causing high training costs for the model. To address these issues, we propose a path planning method, MAPPOHR, which combines heuristic search, empirical rules, and multi-agent reinforcement learning. The method consists of two layers: a real-time planner based on the multi-agent reinforcement learning algorithm, MAPPO, which embeds empirical rules in the action output layer and reward functions, and a heuristic search planner used to create a global guiding path. During movement, the heuristic search planner replans new paths based on the instructions of the real-time planner. We tested our method in 10 different conflict scenarios. The experiments show that the planning performance of MAPPOHR is better than that of existing learning and heuristic methods. Due to the utilization of empirical knowledge and heuristic search, the learning efficiency of MAPPOHR is higher than that of existing learning methods.

翻译：在动态环境中的多机器人路径规划是一个极具挑战性的经典问题。在移动过程中，机器人需要避免与其他移动机器人发生碰撞，同时最小化其行驶距离。以往针对该问题的方法，要么使用启发式搜索方法连续重新规划路径以避免冲突，要么基于学习方法选择合适的避碰策略。前者可能因频繁重新规划导致较长的行驶距离，后者则可能因样本探索和利用率低导致学习效率低下，并造成模型训练成本高昂。为解决这些问题，我们提出了一种路径规划方法MAPPOHR，该方法结合了启发式搜索、经验规则与多智能体强化学习。该方法由两层组成：基于多智能体强化学习算法MAPPO的实时规划器，该规划器在动作输出层和奖励函数中嵌入了经验规则；以及用于创建全局引导路径的启发式搜索规划器。在移动过程中，启发式搜索规划器根据实时规划器的指令重新规划新路径。我们在10种不同的冲突场景中测试了该方法。实验表明，MAPPOHR的规划性能优于现有的学习方法和启发式方法。由于利用了经验知识和启发式搜索，MAPPOHR的学习效率高于现有的学习方法。