Accurate control of autonomous marine robots still poses challenges due to the complex dynamics of the environment. In this paper, we propose a Deep Refinement Learning (DRL) approach to train a controller for autonomous surface vessel (ASV) trajectory tracking and compare its performance with an advanced nonlinear model predictive controller (NMPC) in real environments. Taking into account environmental disturbances (e.g., wind, waves, and currents), noisy measurements, and non-ideal actuators presented in the physical ASV, several effective reward functions for DRL tracking control policies are carefully designed. The control policies were trained in a simulation environment with diverse tracking trajectories and disturbances. The performance of the DRL controller has been verified and compared with the NMPC in both simulations with model-based environmental disturbances and in natural waters. Simulations show that the DRL controller has 53.33% lower tracking error than that of NMPC. Experimental results further show that, compared to NMPC, the DRL controller has 35.51% lower tracking error, indicating that DRL controllers offer better disturbance rejection in river environments than NMPC.

翻译：针对复杂环境动力学带来的挑战，自主海洋机器人的精确控制仍存在问题。本文提出一种深度精化学习（DRL）方法，用于训练自主水面艇（ASV）轨迹跟踪控制器，并在真实环境中将其性能与先进非线性模型预测控制器（NMPC）进行对比。考虑到物理ASV中的环境扰动（如风、浪、流）、噪声测量和非理想执行器，我们精心设计了若干面向DRL跟踪控制策略的有效奖励函数。在包含多种跟踪轨迹和扰动的仿真环境中对控制策略进行了训练。在基于模型的环境扰动仿真及自然水域中，验证了DRL控制器的性能，并与NMPC进行了比较。仿真结果表明，DRL控制器的跟踪误差比NMPC低53.33%。实验结果进一步显示，与NMPC相比，DRL控制器的跟踪误差降低了35.51%，表明在河流环境中DRL控制器比NMPC具有更优的抗扰动能力。