Deep Reinforcement Learning (Deep-RL) techniques for motion control have been continuously used to deal with decision-making problems for a wide variety of robots. Previous works showed that Deep-RL can be applied to perform mapless navigation, including the medium transition of Hybrid Unmanned Aerial Underwater Vehicles (HUAUVs). These are robots that can operate in both air and water media, with future potential for rescue tasks in robotics. This paper presents new approaches based on the state-of-the-art Double Critic Actor-Critic algorithms to address the navigation and medium transition problems for a HUAUV. We show that double-critic Deep-RL with Recurrent Neural Networks using range data and relative localization solely improves the navigation performance of HUAUVs. Our DoCRL approaches achieved better navigation and transitioning capability, outperforming previous approaches.

翻译：深度强化学习（Deep-RL）运动控制技术已持续用于解决多种机器人的决策问题。先前研究表明，深度强化学习可应用于无地图导航任务，包括混合式无人空中水下机器人（HUAUV）的介质过渡。这类机器人能同时在空气和水两种介质中作业，未来在救援机器人领域具有应用潜力。本文提出基于最先进的双评论家Actor-Critic算法的新方法，用于解决HUAUV的导航与介质过渡问题。研究表明，采用循环神经网络的双评论家深度强化学习仅通过距离数据和相对定位即可显著提升HUAUV的导航性能。我们提出的DoCRL方法在导航与过渡能力上均优于先前方法，取得了更优表现。