Multi-Agent Reinforcement Learning (MARL) has enjoyed significant recent progress thanks, in part, to the integration of deep learning techniques for modeling interactions in complex environments. This is naturally starting to benefit multi-robot systems (MRS) in the form of multi-robot RL (MRRL). However, existing infrastructure to train and evaluate policies predominantly focus on the challenges of coordinating virtual agents, and ignore characteristics important to robotic systems. Few platforms support realistic robot dynamics, and fewer still can evaluate Sim2Real performance of learned behavior. To address these issues, we contribute MARBLER: Multi-Agent RL Benchmark and Learning Environment for the Robotarium. MARBLER offers a robust and comprehensive evaluation platform for MRRL by marrying Georgia Tech's Robotarium (which enables rapid deployment on physical MRS) and OpenAI's Gym interface (which facilitates standardized use of modern learning algorithms). MARBLER offers a highly controllable environment with realistic dynamics, including barrier certificate-based obstacle avoidance. It allows anyone across the world to train and deploy MRRL algorithms on a physical testbed with reproducibility. Further, we introduce five novel scenarios inspired by common challenges in MRS and provide support for new custom scenarios. Finally, we use MARBLER to evaluate popular MARL algorithms and provide insights into their suitability for MRRL. In summary, MARBLER can be a valuable tool to the MRS research community by facilitating comprehensive and standardized evaluation of learning algorithms on realistic simulations and physical hardware. Links to our open-source framework and videos of real-world experiments can be found at https://shubhlohiya.github.io/MARBLER/.

翻译：摘要：多智能体强化学习（MARL）通过引入深度学习技术建模复杂环境中的交互，近期取得了显著进展。这自然开始以多机器人强化学习（MRRL）的形式惠及多机器人系统（MRS）。然而，现有训练和评估策略的基础设施主要集中在虚拟智能体协调的挑战上，忽视了机器人系统的重要特性。少数平台支持真实的机器人动力学，能够评估所学行为的Sim2Real性能的平台更是凤毛麟角。为解决这些问题，我们提出了MARBLER：面向Robotarium的多智能体RL基准测试与学习环境。MARBLER结合了佐治亚理工学院的Robotarium（支持在物理MRS上快速部署）与OpenAI的Gym接口（促进现代学习算法的标准化使用），为MRRL提供了稳健且全面的评估平台。该平台具备具有真实动力学特性的高度可控环境，包括基于屏障证书的避障功能，允许全球任意用户以可复现的方式在物理测试平台上训练和部署MRRL算法。此外，我们引入了五个受MRS常见挑战启发的新场景，并支持自定义场景的扩展。最后，我们利用MARBLER评估了主流MARL算法，并深入分析了其对MRRL的适用性。总之，MARBLER通过促进在真实仿真与物理硬件上进行全面且标准化的学习算法评估，可成为MRS研究社区的重要工具。开源框架链接及真实世界实验视频请访问 https://shubhlohiya.github.io/MARBLER/。