For decades, people have been seeking for fishlike flapping motions that can realize underwater propulsion with low energy cost. Complexity of the nonstationary flow field around the flapping body makes this problem very difficult. In earlier studies, motion patterns are usually prescribed as certain periodic functions which constrains the following optimization process in a small subdomain of the whole motion space. In this work, to avoid this motion constraint, a variational autoencoder (VAE) is designed to compress various flapping motions into a simple action vector. Then we let a flapping airfoil continuously interact with water tunnel environment and adjust its action accordingly through a reinforcement learning (RL) framework. By this automatic close-looped experiment, we obtain several motion patterns that can result in high hydrodynamic efficiency comparing to pure harmonic motions with the same thrust level. And we find that, after numerous trials and errors, RL trainings in current experiment always converge to motion patterns that are close to harmonic motions. In other words, current work proves that harmonic motion with appropriate amplitude and frequency is always an optimal choice for efficient underwater propulsion. Furthermore, the RL framework proposed here can be also extended to the study of other complex swimming problems, which might pave the way for the creation of a robotic fish that can swim like a real fish.

翻译：数十年来，人们一直致力于寻找能够实现低能耗水下推进的仿鱼类扑翼运动。扑动体周围非定常流场的复杂性使这一问题极具挑战。早期研究通常将运动模式预设为特定周期函数，这导致后续优化过程被限制在整个运动空间的一小片区域内。为避免这种运动约束，本研究设计了一个变分自编码器将多种扑翼运动压缩为简单的动作向量。随后让扑动翼型与水流隧道环境持续交互，并通过强化学习框架自适应调整动作。通过这种自动闭环实验，我们获得了若干运动模式——与同推力水平的纯简谐运动相比，这些模式能实现更高的水动力效率。实验发现，经过大量试错后，当前实验中的强化学习训练始终收敛于接近简谐运动的运动模式。换言之，本研究证明了合适的振幅与频率的简谐运动始终是高效水下推进的优化选择。此外，本文提出的强化学习框架还可推广至其他复杂游泳问题的研究，这为创造能像真实鱼类般游动的仿生机器鱼奠定了基础。