Markov games with coupling constraints model constrained dynamical decision-making involving self-interested agents, where the feasibility of an individual agent's strategy depends on the joint strategies of the others. Such games arise in numerous real-world applications involving safety requirements and budget caps, for example, in environmental management, electricity markets, and transportation systems. In unconstrained dynamical decision-making, the correlated equilibrium has emerged as a desired solution concept due to its computational tractability and amenability to learning algorithms. Understanding how coupling constraints shape correlated equilibria is a crucial step towards computing solutions in constrained Markov games. In this paper, we formalize and characterize the notion of constrained correlated equilibria for Markov games, defined as feasible joint policies where any unilateral deviation is either unprofitable or infeasible. Building on this characterization, we further study existence conditions for constrained correlated equilibria. In particular, we provide a novel existence proof of such equilibria in Markov games with coupling constraints.

翻译：耦合约束下的马尔可夫博弈模型刻画了涉及自利智能体的受约束动态决策过程，其中个体智能体策略的可行性取决于其他智能体的联合策略。此类博弈广泛存在于涉及安全要求和预算上限的实际应用场景中，例如环境管理、电力市场和交通系统。在无约束动态决策问题中，关联均衡因其计算可处理性及对学习算法的适应性而成为理想的解概念。理解耦合约束如何影响关联均衡，是求解受约束马尔可夫博弈的关键步骤。本文形式化并表征了马尔可夫博弈中约束关联均衡的概念，将其定义为满足任意单边偏离均不可获利或不可行的可行联合策略。基于该表征，我们进一步研究了约束关联均衡的存在性条件，特别地，为耦合约束条件下马尔可夫博弈中此类均衡的存在性提供了新颖的证明。