Correlated Equilibrium (CE) is a well-established solution concept that captures coordination among agents and enjoys good algorithmic properties. In real-world multi-agent systems, in addition to being in an equilibrium, agents' policies are often expected to meet requirements with respect to safety, and fairness. Such additional requirements can often be expressed in terms of the state density which measures the state-visitation frequencies during the course of a game. However, existing CE notions or CE-finding approaches cannot explicitly specify a CE with particular properties concerning state density; they do so implicitly by either modifying reward functions or using value functions as the selection criteria. The resulting CE may thus not fully fulfil the state-density requirements. In this paper, we propose Density-Based Correlated Equilibria (DBCE), a new notion of CE that explicitly takes state density as selection criterion. Concretely, we instantiate DBCE by specifying different state-density requirements motivated by real-world applications. To compute DBCE, we put forward the Density Based Correlated Policy Iteration algorithm for the underlying control problem. We perform experiments on various games where results demonstrate the advantage of our CE-finding approach over existing methods in scenarios with state-density concerns.

翻译：相关均衡（CE）是一种成熟的解概念，能够捕捉智能体之间的协调性，并具有良好的算法性质。在现实世界的多智能体系统中，除了达到均衡之外，智能体的策略通常还需满足安全性和公平性等要求。此类附加要求通常可通过状态密度来表述，该密度衡量博弈过程中状态访问的频率。然而，现有的CE概念或CE求解方法无法显式指定具有特定状态密度性质的CE；它们通过修改奖励函数或使用价值函数作为选择标准来隐式实现这一点。由此产生的CE可能无法完全满足状态密度要求。本文提出基于密度的相关均衡（DBCE），这是一种显式以状态密度为选择标准的新型CE概念。具体而言，我们通过指定受现实应用启发的不同状态密度要求来实例化DBCE。为计算DBCE，我们针对底层控制问题提出了基于密度的相关策略迭代算法。我们在各种博弈上进行实验，结果表明在具有状态密度关注的场景中，我们的CE求解方法相比现有方法具有优势。