Analysis of invasive sports such as soccer is challenging because the game situation changes continuously in time and space, and multiple agents individually recognize the game situation and make decisions. Previous studies using deep reinforcement learning have often considered teams as a single agent and valued the teams and players who hold the ball in each discrete event. Then it was challenging to value the actions of multiple players, including players far from the ball, in a spatiotemporally continuous state space. In this paper, we propose a method of valuing possible actions for on- and off-ball soccer players in a single holistic framework based on multi-agent deep reinforcement learning. We consider a discrete action space in a continuous state space that mimics that of Google research football and leverages supervised learning for actions in reinforcement learning. In the experiment, we analyzed the relationships with conventional indicators, season goals, and game ratings by experts, and showed the effectiveness of the proposed method. Our approach can assess how multiple players move continuously throughout the game, which is difficult to be discretized or labeled but vital for teamwork, scouting, and fan engagement.

翻译：对足球等侵入性运动进行分析颇具挑战性，因为比赛局势在时间和空间上持续变化，且多个智能体各自识别比赛局势并做出决策。以往基于深度强化学习的研究通常将球队视为单一智能体，并在每个离散事件中对控球队及控球球员进行价值评估。因此，在时空连续的状态空间中，难以对包含远离足球球员在内的多个球员行动进行价值评估。本文提出一种基于多智能体深度强化学习的统一整体框架，用于评估足球场上与无球球员可能采取的行动。我们参照谷歌研究足球项目，在连续状态空间中构建离散动作空间，并利用强化学习中的监督学习辅助行动决策。实验分析了该方法与传统指标、赛季进球数及专家比赛评分的关联性，验证了该方法的有效性。本方法能够评估整场比赛中多名球员的连续移动——这种移动难以离散化或标注，但对团队协作、球探观察及球迷互动至关重要。