Independent learners are agents that employ single-agent algorithms in multi-agent systems, intentionally ignoring the effect of other strategic agents. This paper studies mean-field games from a decentralized learning perspective, with two primary objectives: (i) to identify structure that can guide algorithm design, and (ii) to understand the emergent behaviour in systems of independent learners. We study a new model of partially observed mean-field games with finitely many players, local action observability, and a general observation channel for partial observations of the global state. Specific observation channels considered include (a) global observability, (b) local and mean-field observability, (c) local and compressed mean-field observability, and (d) only local observability. We establish conditions under which the control problem of a given agent is equivalent to a fully observed MDP, as well as conditions under which the control problem is equivalent only to a POMDP. Building on the connection to MDPs, we prove the existence of perfect equilibrium among memoryless stationary policies under mean-field observability. Leveraging the connection to POMDPs, we prove convergence of learning iterates obtained by independent learning agents under any of the aforementioned observation channels. We interpret the limiting values as subjective value functions, which an agent believes to be relevant to its control problem. These subjective value functions are then used to propose subjective Q-equilibrium, a new solution concept for partially observed n-player mean-field games, whose existence is proved under mean-field or global observability.We provide a decentralized learning algorithm for partially observed n-player mean-field games, and we show that it drives play to subjective Q-equilibrium by adapting the recently developed theory of satisficing paths to allow for subjectivity.

翻译：独立学习者是在多智能体系统中采用单智能体算法的智能体，有意忽略其他策略智能体的影响。本文从分布式学习视角研究平均场博弈，主要目标有二：（i）识别可指导算法设计的结构特征，（ii）理解独立学习者系统中的涌现行为。我们研究了一种新型部分观测平均场博弈模型，该模型包含有限数量智能体、局部动作可观测性，以及用于全局状态部分观测的通用观测通道。具体考虑的观测通道包括：（a）全局可观测性，（b）局部与平均场可观测性，（c）局部与压缩平均场可观测性，以及（d）仅局部可观测性。我们建立了给定智能体控制问题等价于完全观测马尔可夫决策过程的条件，以及控制问题仅等价于部分观测马尔可夫决策过程的条件。基于与马尔可夫决策过程的联系，我们证明了在平均场可观测性下无记忆平稳策略完美均衡的存在性。借助部分观测马尔可夫决策过程的联系，我们证明了在上述任意观测通道下独立学习智能体获得的学习迭代的收敛性。我们将极限值解释为主观价值函数，即智能体认为与其控制问题相关的函数。这些主观价值函数进而用于提出主观Q均衡——一种面向部分观测n智能体平均场博弈的新解概念，并在平均场或全局可观测性下证明其存在性。我们为部分观测n智能体平均场博弈提出了一种分布式学习算法，并通过调整近期发展的满意路径理论以纳入主观性，证明该算法能驱动博弈收敛至主观Q均衡。