Navigating efficiently across vortical flow fields presents a significant challenge in various robotic applications. The dynamic and unsteady nature of vortical flows often disturbs the control of underwater robots, complicating their operation in hydrodynamic environments. Conventional control methods, which depend on accurate modeling, fail in these settings due to the complexity of fluid-structure interactions (FSI) caused by unsteady hydrodynamics. This study proposes a deep reinforcement learning (DRL) algorithm, trained in a data-driven manner, to enable efficient navigation of a robotic fish swimming across vortical flows. Our proposed algorithm incorporates the LSTM architecture and uses several recent consecutive observations as the state to address the issue of partial observation, often due to sensor limitations. We present a numerical study of navigation within a Karman vortex street, created by placing a stationary cylinder in a uniform flow, utilizing the immersed boundary-lattice Boltzmann method (IB-LBM). The aim is to train the robotic fish to discover efficient navigation policies, enabling it to reach a designated target point across the Karman vortex street from various initial positions. After training, the fish demonstrates the ability to rapidly reach the target from different initial positions, showcasing the effectiveness and robustness of our proposed algorithm. Analysis of the results reveals that the robotic fish can leverage velocity gains and pressure differences induced by the vortices to reach the target, underscoring the potential of our proposed algorithm in enhancing navigation in complex hydrodynamic environments.

翻译：在各类机器人应用中，实现穿越涡流场的高效导航是一项重大挑战。涡流的动态非定常特性常干扰水下机器人的控制，使其在流体动力学环境中的操作复杂化。传统控制方法依赖精确建模，但由于非定常流体动力学引起的流固耦合（FSI）复杂性，这些方法在此类场景中往往失效。本研究提出一种基于数据驱动训练的深度强化学习（DRL）算法，以实现仿生机器鱼在涡流中游泳的高效导航。所提算法采用LSTM架构，并以最近连续观测值作为状态输入，以解决因传感器限制导致的局部观测问题。我们通过浸没边界-格子玻尔兹曼方法（IB-LBM），对置于均匀流场中的静止圆柱产生的卡门涡街进行了导航数值研究。目标是训练仿生机器鱼探索高效导航策略，使其能够从不同初始位置穿越卡门涡街抵达指定目标点。训练完成后，该机器鱼展现出从不同起始位置快速抵达目标的能力，证明了所提算法的有效性与鲁棒性。结果分析表明，仿生机器鱼能够利用涡流诱导的速度增益与压差抵达目标，这凸显了所提算法在增强复杂流体动力学环境中导航能力的潜力。