In marine operations underwater manipulators play a primordial role. However, due to uncertainties in the dynamic model and disturbances caused by the environment, low-level control methods require great capabilities to adapt to change. Furthermore, under position and torque constraints the requirements for the control system are greatly increased. Reinforcement learning is a data driven control technique that can learn complex control policies without the need of a model. The learning capabilities of these type of agents allow for great adaptability to changes in the operative conditions. In this article we present a novel reinforcement learning low-level controller for the position control of an underwater manipulator under torque and position constraints. The reinforcement learning agent is based on an actor-critic architecture using sensor readings as state information. Simulation results using the Reach Alpha 5 underwater manipulator show the advantages of the proposed control strategy.

翻译：在海洋作业中,水下操纵器起一种首要作用,然而,由于动态模型的不确定性和环境造成的扰动,低水平控制方法需要巨大的适应能力以适应变化;此外,在位置和压力限制下,控制系统的要求大大增加;强化学习是一种数据驱动控制技术,可以学习复杂的控制政策,而不需要模型;这些类型的制剂的学习能力使得极能适应操作条件的变化;在本篇文章中,我们介绍了一个新的强化学习低水平控制器,用于在压力和姿势限制下控制水下操纵器的位置。强化学习剂以一个以传感器读读为状态信息的行为者-批评结构为基础。使用Rach Alpha 5 水下操纵器的模拟结果显示了拟议控制战略的优势。