This paper proposes a new notion of Markov $\alpha$-potential games to study Markov games. Two important classes of practically significant Markov games, Markov congestion games and the perturbed Markov team games, are analyzed in this framework of Markov $\alpha$-potential games, with explicit characterization of the upper bound for $\alpha$ and its relation to game parameters. Moreover, any maximizer of the $\alpha$-potential function is shown to be an $\alpha$-stationary Nash equilibrium of the game. Furthermore, two algorithms for the Nash regret analysis, namely the projected gradient-ascent algorithm and the sequential maximum improvement algorithm, are presented and corroborated by numerical experiments.

翻译：本文提出了一种新的马尔可夫$\alpha$-势博弈概念，用于研究马尔可夫博弈。在该框架下，分析了两类具有实际重要性的马尔可夫博弈——马尔可夫拥塞博弈和受扰马尔可夫团队博弈，明确了$\alpha$的上界及其与博弈参数的关系。此外，研究证明了$\alpha$-势函数的任何最大化者都是博弈的一个$\alpha$-平稳纳什均衡。进一步地，本文提出了两种用于纳什遗憾分析的算法，即投影梯度上升算法和序列最大改进算法，并通过数值实验验证了其有效性。