Coverage path planning (CPP) is the problem of finding a path that covers the entire free space of a confined area, with applications ranging from robotic lawn mowing and vacuum cleaning, to demining and search-and-rescue tasks. While offline methods can find provably complete, and in some cases optimal, paths for known environments, their value is limited in online scenarios where the environment is not known beforehand. In this case, the path needs to be planned online while mapping the environment. We investigate how suitable reinforcement learning is for this challenging problem, and analyze the involved components required to efficiently learn coverage paths, such as action space, input feature representation, neural network architecture, and reward function. Compared to existing classical methods, this approach allows for a flexible path space, and enables the agent to adapt to specific environment dynamics. In addition to local sensory inputs for acting on short-term obstacle detections, we propose to use egocentric maps in multiple scales based on frontiers. This allows the agent to plan a long-term path in large-scale environments with feasible computational and memory complexity. Furthermore, we propose a novel total variation reward term for guiding the agent not to leave small holes of non-covered free space. To validate the effectiveness of our approach, we perform extensive experiments in simulation with a 2D ranging sensor on different variations of the CPP problem, surpassing the performance of both previous RL-based approaches and highly specialized methods.

翻译：覆盖路径规划（CPP）是指在受限区域内寻找一条能够遍历整个自由空间的路径，其应用涵盖机器人割草、真空清洁、排雷及搜索救援等任务。离线方法虽能为已知环境提供可证明完备乃至最优的路径，但在环境信息未知的在线场景中其价值有限——此时需在构建环境地图的同时在线规划路径。本研究探索强化学习对该挑战性问题的适用性，深入分析高效学习覆盖路径所需的关键组件（如动作空间、输入特征表示、神经网络架构及奖励函数）。与现有经典方法相比，该方法具备灵活的路径空间，且能使智能体适应特定环境动态特性。除基于局部传感器输入进行短期障碍物探测外，我们提出采用基于前沿的多尺度自我定位地图，使智能体能够以可行的计算与内存复杂度在大型环境中规划长期路径。此外，我们创新性地提出全变分奖励项，引导智能体避免遗留未覆盖自由空间中的小型空洞。为验证方法有效性，我们在二维测距传感器环境下对CPP问题的多种变体进行了大规模仿真实验，性能超越此前基于强化学习的方法及高度专业化方法。