Even though Google Research Football (GRF) was initially benchmarked and studied as a single-agent environment in its original paper, recent years have witnessed an increasing focus on its multi-agent nature by researchers utilizing it as a testbed for Multi-Agent Reinforcement Learning (MARL). However, the absence of standardized environment settings and unified evaluation metrics for multi-agent scenarios hampers the consistent understanding of various studies. Furthermore, the challenging 5-vs-5 and 11-vs-11 full-game scenarios have received limited thorough examination due to their substantial training complexities. To address these gaps, this paper extends the original environment by not only standardizing the environment settings and benchmarking cooperative learning algorithms across different scenarios, including the most challenging full-game scenarios, but also by discussing approaches to enhance football AI from diverse perspectives and introducing related research tools. Specifically, we provide a distributed and asynchronous population-based self-play framework with diverse pre-trained policies for faster training, two football-specific analytical tools for deeper investigation, and an online leaderboard for broader evaluation. The overall expectation of this work is to advance the study of Multi-Agent Reinforcement Learning on Google Research Football environment, with the ultimate goal of benefiting real-world sports beyond virtual games.

翻译：尽管谷歌研究足球（GRF）在最初的研究中被设定为单智能体环境进行基准测试与分析，但近年来研究者们日益关注其多智能体特性，并将其作为多智能体强化学习（MARL）的测试平台。然而，多智能体场景中标准环境设置与统一评估指标的缺失，阻碍了对各类研究结果的一致性理解。此外，由于5对5和11对11全场比赛场景的训练复杂度极高，目前鲜有对其进行的深入系统研究。为弥补上述不足，本文不仅通过标准化环境设置并在不同场景（包括最具挑战性的全场比赛场景）下对合作学习算法进行基准测试，还从多种视角探讨了增强足球人工智能的途径并引入相关研究工具。具体而言，我们提出了一个基于分布式异步种群自对弈框架，其配备多样化预训练策略以加速训练过程；开发了两套足球专用分析工具用于深度探究；并构建在线排行榜以支持更广泛的评估。本研究旨在推动基于谷歌研究足球环境的多智能体强化学习研究，最终目标是为虚拟游戏之外的真实体育领域带来实际效益。