Accurate control of autonomous marine robots still poses challenges due to the complex dynamics of the environment. In this paper, we propose a Deep Reinforcement Learning (DRL) approach to train a controller for autonomous surface vessel (ASV) trajectory tracking and compare its performance with an advanced nonlinear model predictive controller (NMPC) in real environments. Taking into account environmental disturbances (e.g., wind, waves, and currents), noisy measurements, and non-ideal actuators presented in the physical ASV, several effective reward functions for DRL tracking control policies are carefully designed. The control policies were trained in a simulation environment with diverse tracking trajectories and disturbances. The performance of the DRL controller has been verified and compared with the NMPC in both simulations with model-based environmental disturbances and in natural waters. Simulations show that the DRL controller has 53.33% lower tracking error than that of NMPC. Experimental results further show that, compared to NMPC, the DRL controller has 35.51% lower tracking error, indicating that DRL controllers offer better disturbance rejection in river environments than NMPC.

翻译：由于环境的复杂动力学特性，自主海洋机器人的精确控制仍面临挑战。本文提出一种基于深度强化学习的控制器训练方法，用于自主水面船舶的轨迹跟踪，并在真实环境中将其性能与先进非线性模型预测控制器进行对比。考虑物理自主水面船舶中的环境扰动（如风、浪、流）、噪声测量及非理想执行器，我们精心设计了多种用于深度强化学习跟踪控制策略的有效奖励函数。控制策略在包含多种跟踪轨迹与扰动的仿真环境中完成训练。通过基于模型的环境扰动仿真测试与自然水域实船实验，深度强化学习控制器的性能得到验证并与非线性模型预测控制器进行了对比。仿真结果显示，深度强化学习控制器的跟踪误差相比非线性模型预测控制器降低53.33%。实验结果进一步表明，相较于非线性模型预测控制器，深度强化学习控制器的跟踪误差降低35.51%，说明深度强化学习控制器在河流环境中具有更优的扰动抑制能力。