Multi-agent reinforcement learning (MARL) has made significant progress, largely fueled by the development of specialized testbeds that enable systematic evaluation of algorithms in controlled yet challenging scenarios. However, existing testbeds often focus on purely virtual simulations or limited robot morphologies such as robotic arms, quadrupeds, and humanoids, leaving high-mobility platforms with real-world physical constraints like drones underexplored. To bridge this gap, we present VolleyBots, a new MARL testbed where multiple drones cooperate and compete in the sport of volleyball under physical dynamics. VolleyBots features a turn-based interaction model under volleyball rules, a hierarchical decision-making process that combines motion control and strategic play, and a high-fidelity simulation for seamless sim-to-real transfer. We provide a comprehensive suite of tasks ranging from single-drone drills to multi-drone cooperative and competitive tasks, accompanied by baseline evaluations of representative MARL and game-theoretic algorithms. Results in simulation show that while existing algorithms handle simple tasks effectively, they encounter difficulty in complex tasks that require both low-level control and high-level strategy. We further demonstrate zero-shot deployment of a simulation-learned policy to real-world drones, highlighting VolleyBots' potential to propel MARL research involving agile robotic platforms. The project page is at https://sites.google.com/view/volleybots/home.

翻译：多智能体强化学习（MARL）已取得显著进展，这在很大程度上得益于专用测试平台的发展，这些平台能够在受控且具有挑战性的场景中对算法进行系统评估。然而，现有的测试平台通常侧重于纯虚拟仿真或有限的机器人形态，如机械臂、四足机器人和仿人机器人，而对具有现实物理约束的高机动性平台（如无人机）的探索尚不充分。为弥补这一空白，我们提出了VolleyBots，这是一个新的MARL测试平台，其中多个无人机在物理动力学约束下，于排球运动中展开合作与竞争。VolleyBots具备以下特点：基于排球规则的回合制交互模型、融合运动控制与策略博弈的层次化决策过程，以及支持无缝仿真到现实迁移的高保真仿真环境。我们提供了一套全面的任务集，涵盖从单无人机基础训练到多无人机协作与对抗任务，并对代表性的MARL及博弈论算法进行了基线评估。仿真结果表明，现有算法能有效处理简单任务，但在需要底层运动控制与高层策略协同的复杂任务中则面临困难。我们进一步展示了将仿真学习到的策略零样本部署到真实无人机上，凸显了VolleyBots在推动涉及敏捷机器人平台的MARL研究方面的潜力。项目页面位于 https://sites.google.com/view/volleybots/home。