This paper presents multi-agent reinforcement learning frameworks for the low-level control of a quadrotor UAV. While single-agent reinforcement learning has been successfully applied to quadrotors, training a single monolithic network is often data-intensive and time-consuming. To address this, we decompose the quadrotor dynamics into the translational dynamics and the yawing dynamics, and assign a reinforcement learning agent to each part for efficient training and performance improvements. The proposed multi-agent framework for quadrotor low-level control that leverages the underlying structures of the quadrotor dynamics is a unique contribution. Further, we introduce regularization terms to mitigate steady-state errors and to avoid aggressive control inputs. Through benchmark studies with sim-to-sim transfer, it is illustrated that the proposed multi-agent reinforcement learning substantially improves the convergence rate of the training and the stability of the controlled dynamics.

翻译：本文提出了用于四旋翼无人机底层控制的多智能体强化学习框架。尽管单智能体强化学习已成功应用于四旋翼无人机，但训练单一整体网络往往需要大量数据和较长时间。为解决这一问题，我们将四旋翼动力学分解为平移动力学和偏航动力学两部分，并为每部分分配一个强化学习智能体，以实现高效训练和性能提升。所提出的多智能体框架利用四旋翼动力学的内在结构，在底层控制领域具有独特贡献。此外，我们引入正则化项来降低稳态误差并避免激进的控制输入。通过仿真到仿真的基准迁移研究，表明所提出的多智能体强化学习显著提高了训练收敛速度和控制动力学的稳定性。