This paper proposes a new framework of Markov $\alpha$-potential games to study Markov games. In this new framework, Markov games are shown to be Markov $\alpha$-potential games, and the existence of an associated $\alpha$-potential function is established. Any optimizer of an $\alpha$-potential function is shown to be an $\alpha$-stationary NE. Two important classes of practically significant Markov games, Markov congestion games and the perturbed Markov team games, are studied via this framework of Markov $\alpha$-potential games, with explicit characterization of an upper bound for $\alpha$ and its relation to game parameters. Additionally, a semi-infinite linear programming based formulation is presented to obtain an upper bound for $\alpha$ for any Markov game. Furthermore, two equilibrium approximation algorithms, namely the projected gradient-ascent algorithm and the sequential maximum improvement algorithm, are presented along with their Nash regret analysis, and corroborated by numerical experiments.

翻译：本文提出了一种新的马尔可夫$α$-势博弈框架用以研究马尔可夫博弈。在该新框架下，证明了马尔可夫博弈可转化为马尔可夫$α$-势博弈，并建立了关联$α$-势函数的存在性。证明了$α$-势函数的任意优化器均为$α$-平稳纳什均衡。通过该马尔可夫$α$-势博弈框架，研究了马尔可夫拥塞博弈与扰动马尔可夫团队博弈这两类具有重要实际意义的马尔可夫博弈，明确刻画了$α$的上界及其与博弈参数的关联。此外，提出了一种基于半无限线性规划的表达式，用以获得任意马尔可夫博弈的$α$上界。进一步，给出了两种均衡近似算法——投影梯度上升算法与序贯最大改进算法，并对其进行了纳什遗憾分析，数值实验验证了算法的有效性。