By leveraging the underlying structures of the quadrotor dynamics, we propose multi-agent reinforcement learning frameworks to innovate the low-level control of a quadrotor, where independent agents operate cooperatively to achieve a common goal. While single-agent reinforcement learning has been successfully applied in quadrotor controls, training a large monolithic network is often data-intensive and time-consuming. Moreover, achieving agile yawing control remains a significant challenge due to the strongly coupled nature of the quadrotor dynamics. To address this, we decompose the quadrotor dynamics into translational and yawing components and assign collaborative reinforcement learning agents to each part to facilitate more efficient training. Additionally, we introduce regularization terms to mitigate steady-state errors and prevent excessive maneuvers. Benchmark studies, including sim-to-sim transfer verification, demonstrate that our proposed training schemes substantially improve the convergence rate of training, while enhancing flight control performance and stability compared to traditional single-agent approaches.

翻译：通过利用四旋翼动力学底层结构，我们提出多智能体强化学习框架以实现四旋翼底层控制的创新，其中独立智能体协同运作以达到共同目标。尽管单智能体强化学习已成功应用于四旋翼控制，但训练大规模单一网络通常需要大量数据且耗时较长。此外，由于四旋翼动力学的强耦合特性，实现敏捷偏航控制仍是重大挑战。为此，我们将四旋翼动力学分解为平移与偏航分量，并为各分量分配协作式强化学习智能体以提升训练效率。同时引入正则化项来抑制稳态误差并避免过度机动。包含仿真环境间迁移验证的基准研究表明，与传统单智能体方法相比，我们提出的训练方案显著提高了训练收敛速度，并增强了飞行控制性能与稳定性。