In recent years, interest in autonomous shipping in urban waterways has increased significantly due to the trend of keeping cars and trucks out of city centers. Classical approaches such as Frenet frame based planning and potential field navigation often require tuning of many configuration parameters and sometimes even require a different configuration depending on the situation. In this paper, we propose a novel path planning approach based on reinforcement learning called Model Predictive Reinforcement Learning (MPRL). MPRL calculates a series of waypoints for the vessel to follow. The environment is represented as an occupancy grid map, allowing us to deal with any shape of waterway and any number and shape of obstacles. We demonstrate our approach on two scenarios and compare the resulting path with path planning using a Frenet frame and path planning based on a proximal policy optimization (PPO) agent. Our results show that MPRL outperforms both baselines in both test scenarios. The PPO based approach was not able to reach the goal in either scenario while the Frenet frame approach failed in the scenario consisting of a corner with obstacles. MPRL was able to safely (collision free) navigate to the goal in both of the test scenarios.

翻译：近年来，由于将汽车和卡车限制在城市中心之外的趋势，城市水道中自主航运的兴趣显著增加。传统的路径规划方法，如基于Frenet框架的规划和势场导航，通常需要调整大量配置参数，有时甚至需要根据具体情况采用不同的配置。本文提出了一种基于强化学习的新型路径规划方法——模型预测强化学习（MPRL）。MPRL计算一系列航路点供船舶跟踪。环境被表示为占据网格地图，使我们能够处理任意形状的水道以及任意数量和形状的障碍物。我们在两个场景中演示了该方法，并将所得路径与使用Frenet框架的路径规划及基于近端策略优化（PPO）智能体的路径规划进行了比较。结果表明，MPRL在两个测试场景中均优于两种基线方法。基于PPO的方法在两个场景中均未能到达目标，而基于Frenet框架的方法在包含障碍物的弯道场景中失败。MPRL则能在两个测试场景中安全（无碰撞）地导航至目标。