We present a deep reinforcement learning approach to a classical problem in fluid dynamics, i.e., the reduction of the drag of a bluff body. We cast the problem as a discrete-time control with continuous action space: at each time step, an autonomous agent can set the flow rate of two jets of fluid, positioned at the back of the body. The agent, trained with Proximal Policy Optimization, learns an effective strategy to make the jets interact with the vortexes of the wake, thus reducing the drag. To tackle the computational complexity of the fluid dynamics simulations, which would make the training procedure prohibitively expensive, we train the agent on a coarse discretization of the domain. We provide numerical evidence that a policy trained in this approximate environment still retains good performance when carried over to a denser mesh. Our simulations show a considerable drag reduction with a consequent saving of total power, defined as the sum of the power spent by the control system and of the power of the drag force, amounting to 40% when compared to simulations with the reference bluff body without any jet. Finally, we qualitatively investigate the control policy learnt by the neural network. We can observe that it achieves the drag reduction by learning the frequency of formation of the vortexes and activating the jets accordingly, thus blowing them away off the rear body surface.

翻译：我们提出了一种面向流体动力学经典问题——钝体减阻的深度强化学习方案。将问题建模为连续动作空间的离散时间控制过程：在每个时间步，自主智能体可调控布置于钝体背部的两个流体射流流量。采用近端策略优化训练的智能体习得了使射流有效干预尾流涡旋运动的策略，从而实现减阻目标。为应对流体动力学模拟的高计算复杂度（该问题将导致训练成本过高），我们采用粗网格域离散化方案训练智能体。数值实验表明，在该近似环境中训练的决策策略迁移至精密网格时仍保持良好性能。仿真结果显示，与未配置射流的参考钝体模型相比，该系统实现了显著的阻力降低，同时节约了总功率（控制能耗与阻力功率之和）达40%。最后，我们对神经网络习得的控制策略进行定性分析，观察到其通过识别涡旋形成频率并触发相应射流，将涡旋从钝体背面吹离，从而实现减阻效果。