Multi-agent deep Reinforcement Learning (RL) has made significant progress in developing intelligent game-playing agents in recent years. However, the efficient training of collective robots using multi-agent RL and the transfer of learned policies to real-world applications remain open research questions. In this work, we first develop a comprehensive robotic system, including simulation, distributed learning framework, and physical robot components. We then propose and evaluate reinforcement learning techniques designed for efficient training of cooperative and competitive policies on this platform. To address the challenges of multi-agent sim-to-real transfer, we introduce Out of Distribution State Initialization (OODSI) to mitigate the impact of the sim-to-real gap. In the experiments, OODSI improves the Sim2Real performance by 20%. We demonstrate the effectiveness of our approach through experiments with a multi-robot car competitive game and a cooperative task in real-world settings.

翻译：近年来，多智能体深度强化学习在开发智能博弈智能体方面取得了显著进展。然而，如何利用多智能体强化学习高效训练机器人群体，并将习得策略迁移至现实世界应用，仍是亟待解决的研究课题。本研究首先构建了一套完整的机器人系统，包括仿真环境、分布式学习框架与实体机器人组件。随后，我们提出并评估了专为该平台设计的强化学习技术，旨在实现协作与竞争策略的高效训练。为应对多智能体仿真到现实迁移的挑战，我们引入了分布外状态初始化方法，以缓解仿真与现实差距带来的影响。实验表明，该方法将仿真到现实的性能提升了20%。我们通过多机器人车辆竞争博弈和现实场景协作任务的实验，验证了所提方法的有效性。