Rayleigh-B\'enard convection (RBC) is a recurrent phenomenon in several industrial and geoscience flows and a well-studied system from a fundamental fluid-mechanics viewpoint. However, controlling RBC, for example by modulating the spatial distribution of the bottom-plate heating in the canonical RBC configuration, remains a challenging topic for classical control-theory methods. In the present work, we apply deep reinforcement learning (DRL) for controlling RBC. We show that effective RBC control can be obtained by leveraging invariant multi-agent reinforcement learning (MARL), which takes advantage of the locality and translational invariance inherent to RBC flows inside wide channels. The MARL framework applied to RBC allows for an increase in the number of control segments without encountering the curse of dimensionality that would result from a naive increase in the DRL action-size dimension. This is made possible by the MARL ability for re-using the knowledge generated in different parts of the RBC domain. We show in a case study that MARL DRL is able to discover an advanced control strategy that destabilizes the spontaneous RBC double-cell pattern, changes the topology of RBC by coalescing adjacent convection cells, and actively controls the resulting coalesced cell to bring it to a new stable configuration. This modified flow configuration results in reduced convective heat transfer, which is beneficial in several industrial processes. Therefore, our work both shows the potential of MARL DRL for controlling large RBC systems, as well as demonstrates the possibility for DRL to discover strategies that move the RBC configuration between different topological configurations, yielding desirable heat-transfer characteristics. These results are useful for both gaining further understanding of the intrinsic properties of RBC, as well as for developing industrial applications.

翻译：瑞利-贝纳德对流（RBC）是多种工业与地球物理流动中的常见现象，也是基础流体力学领域研究透彻的系统。然而，通过调节经典RBC构型中底板加热的空间分布等控制手段，对经典控制理论方法而言仍是一项挑战。本研究应用深度强化学习（DRL）控制RBC，证明利用不变性多智能体强化学习（MARL）可实现有效RBC控制——该方法充分利用了宽通道内RBC流动固有的局部性与平移不变性。将MARL框架应用于RBC，可在不因DRL动作维度简单扩展而陷入维数灾难的前提下，增加控制分段数量。这得益于MARL复用RBC域内不同区域生成知识的能力。案例研究表明，MARL-DRL能够发现一种先进控制策略：破坏自发的RBC双涡结构，通过合并相邻对流单元改变RBC拓扑形态，并主动控制合并后的单元使其达到新的稳定构型。这种修改后的流动构型降低了对流换热强度，在众多工业过程中具有重要价值。因此，本研究既展现了MARL-DRL控制大规模RBC系统的潜力，也证明了DRL能够发现推动RBC构型在不同拓扑结构间转换并实现理想传热特性的策略。这些成果既有助于深入理解RBC的内在特性，也为工业应用开发提供了支撑。