The works of (Daskalakis et al., 2009, 2022; Jin et al., 2022; Deng et al., 2023) indicate that computing Nash equilibria in multi-player Markov games is a computationally hard task. This fact raises the question of whether or not computational intractability can be circumvented if one focuses on specific classes of Markov games. One such example is two-player zero-sum Markov games, in which efficient ways to compute a Nash equilibrium are known. Inspired by zero-sum polymatrix normal-form games (Cai et al., 2016), we define a class of zero-sum multi-agent Markov games in which there are only pairwise interactions described by a graph that changes per state. For this class of Markov games, we show that an $\epsilon$-approximate Nash equilibrium can be found efficiently. To do so, we generalize the techniques of (Cai et al., 2016), by showing that the set of coarse-correlated equilibria collapses to the set of Nash equilibria. Afterwards, it is possible to use any algorithm in the literature that computes approximate coarse-correlated equilibria Markovian policies to get an approximate Nash equilibrium.

翻译：(Daskalakis等人，2009, 2022; Jin等人，2022; Deng等人，2023)的研究表明，在多智能体马尔可夫博弈中计算纳什均衡是一项计算上困难的任务。这一事实引发了如下问题：若聚焦于特定类别的马尔可夫博弈，计算复杂性是否可以被规避？其中一个例子是两人零和马尔可夫博弈，已知存在高效计算纳什均衡的方法。受零和多项式矩阵正规型博弈(Cai等人，2016)的启发，我们定义了一类零和多智能体马尔可夫博弈，其中仅存在由每个状态变化的图所描述的成对交互。对于此类马尔可夫博弈，我们证明了可以高效地找到一个$\epsilon$-近似纳什均衡。为此，我们推广了(Cai等人，2016)的技术，展示了粗相关均衡集坍缩为纳什均衡集。随后，可利用文献中任何计算近似粗相关马尔可夫策略均衡的算法，来获得一个近似纳什均衡。