The advent of deep reinforcement learning (DRL) has significantly advanced the field of robotics, particularly in the control and coordination of quadruped robots. However, the complexity of real-world tasks often necessitates the deployment of multi-robot systems capable of sophisticated interaction and collaboration. To address this need, we introduce the Multi-agent Quadruped Environment (MQE), a novel platform designed to facilitate the development and evaluation of multi-agent reinforcement learning (MARL) algorithms in realistic and dynamic scenarios. MQE emphasizes complex interactions between robots and objects, hierarchical policy structures, and challenging evaluation scenarios that reflect real-world applications. We present a series of collaborative and competitive tasks within MQE, ranging from simple coordination to complex adversarial interactions, and benchmark state-of-the-art MARL algorithms. Our findings indicate that hierarchical reinforcement learning can simplify task learning, but also highlight the need for advanced algorithms capable of handling the intricate dynamics of multi-agent interactions. MQE serves as a stepping stone towards bridging the gap between simulation and practical deployment, offering a rich environment for future research in multi-agent systems and robot learning. For open-sourced code and more details of MQE, please refer to https://ziyanx02.github.io/multiagent-quadruped-environment/ .

翻译：深度强化学习（DRL）的出现显著推动了机器人学领域的发展，尤其是在四足机器人的控制与协调方面。然而，现实世界任务的复杂性往往需要部署能够实现复杂交互与协作的多机器人系统。为应对这一需求，我们提出了多智能体四足环境（MQE），这是一个新型平台，旨在促进多智能体强化学习（MARL）算法在真实动态场景中的开发与评估。MQE强调机器人-物体间的复杂交互、层次化策略结构以及反映实际应用的挑战性评估场景。我们在MQE中设计了一系列协作与竞争任务，涵盖从简单协调到复杂对抗交互的范畴，并对当前最先进的MARL算法进行了基准测试。研究结果表明，层次化强化学习可简化任务学习过程，但同时也凸显了对能够处理多智能体交互复杂动态的高级算法的需求。MQE作为连接仿真与实际部署的桥梁，为未来多智能体系统及机器人学习研究提供了丰富的环境。开源代码及MQE更多详情请访问：https://ziyanx02.github.io/multiagent-quadruped-environment/ 。