Deep reinforcement learning (DRL) has found application in numerous use-cases pertaining to flow control. Multi-agent RL (MARL), a variant of DRL, has shown to be more effective than single-agent RL in controlling flows exhibiting locality and translational invariance. We present, for the first time, an implementation of MARL-based control of three-dimensional Rayleigh-B\'enard convection (RBC). Control is executed by modifying the temperature distribution along the bottom wall divided into multiple control segments, each of which acts as an independent agent. Two regimes of RBC are considered at Rayleigh numbers $\mathrm{Ra}=500$ and $750$. Evaluation of the learned control policy reveals a reduction in convection intensity by $23.5\%$ and $8.7\%$ at $\mathrm{Ra}=500$ and $750$, respectively. The MARL controller converts irregularly shaped convective patterns to regular straight rolls with lower convection that resemble flow in a relatively more stable regime. We draw comparisons with proportional control at both $\mathrm{Ra}$ and show that MARL is able to outperform the proportional controller. The learned control strategy is complex, featuring different non-linear segment-wise actuator delays and actuation magnitudes. We also perform successful evaluations on a larger domain than used for training, demonstrating that the invariant property of MARL allows direct transfer of the learnt policy.

翻译：深度强化学习（DRL）已在众多流动控制相关应用中得到验证。多智能体强化学习（MARL）作为DRL的一种变体，在控制具有局部性和平移不变性的流动方面表现出比单智能体强化学习更高的效能。本文首次提出了基于MARL的三维瑞利-贝纳德对流（RBC）控制方法。控制通过调节底部壁面温度分布实现，该壁面被划分为多个控制区段，每个区段作为独立智能体执行控制。研究考虑了瑞利数$\mathrm{Ra}=500$和$750$两种RBC流态。对学习所得控制策略的评估表明，在$\mathrm{Ra}=500$和$750$条件下对流强度分别降低$23.5\%$和$8.7\%$。MARL控制器将不规则的对流形态转化为规则直列状的低强度对流涡卷，其流动特征类似于相对更稳定流态下的流动。我们与两种$\mathrm{Ra}$下的比例控制进行对比，证明MARL能够超越比例控制器的性能。学习得到的控制策略具有复杂性，表现为不同非线性区段间的执行器延迟和执行幅值差异。此外，我们在比训练域更大的计算域上成功进行了评估，证明MARL的不变性特性允许直接将学习策略进行迁移应用。