Learning the behavior of large agent populations is an important task for numerous research areas. Although the field of multi-agent reinforcement learning (MARL) has made significant progress towards solving these systems, solutions for many agents often remain computationally infeasible and lack theoretical guarantees. Mean Field Games (MFGs) address both of these issues and can be extended to Graphon MFGs (GMFGs) to include network structures between agents. Despite their merits, the real world applicability of GMFGs is limited by the fact that graphons only capture dense graphs. Since most empirically observed networks show some degree of sparsity, such as power law graphs, the GMFG framework is insufficient for capturing these network topologies. Thus, we introduce the novel concept of Graphex MFGs (GXMFGs) which builds on the graph theoretical concept of graphexes. Graphexes are the limiting objects to sparse graph sequences that also have other desirable features such as the small world property. Learning equilibria in these games is challenging due to the rich and sparse structure of the underlying graphs. To tackle these challenges, we design a new learning algorithm tailored to the GXMFG setup. This hybrid graphex learning approach leverages that the system mainly consists of a highly connected core and a sparse periphery. After defining the system and providing a theoretical analysis, we state our learning approach and demonstrate its learning capabilities on both synthetic graphs and real-world networks. This comparison shows that our GXMFG learning algorithm successfully extends MFGs to a highly relevant class of hard, realistic learning problems that are not accurately addressed by current MARL and MFG methods.

翻译：学习大规模智能体群体的行为是众多研究领域的重要任务。尽管多智能体强化学习（MARL）领域在解决此类系统方面取得了显著进展，但针对大量智能体的解决方案在计算上往往仍不可行，且缺乏理论保证。平均场博弈（MFG）解决了这两个问题，并可扩展至图论平均场博弈（GMFG）以纳入智能体间的网络结构。尽管具有优势，但GMFG在现实世界的适用性受限于图论仅能刻画稠密图这一事实。由于大多数经验观测到的网络（如幂律图）都呈现出一定程度的稀疏性，因此GMFG框架不足以捕获这类网络拓扑结构。为此，我们提出了一种名为图指数平均场博弈（GXMFG）的新概念，该概念建立在图论中图指数的理论基础上。图指数是稀疏图序列的极限对象，同时具备小世界特性等其他理想特征。由于底层图的丰富稀疏结构，在这类博弈中学习均衡极具挑战性。为应对这些挑战，我们设计了一种专为GXMFG设置的全新学习算法。这种混合图指数学习方法利用了系统主要由高度连通的核心和稀疏外围构成的特点。在定义系统并提供理论分析后，我们阐述了学习方法，并在合成图与真实世界网络上展示了其学习能力。对比结果表明，我们的GXMFG学习算法成功地将MFG扩展至一类当前MARL和MFG方法未能准确处理的高度相关且困难的现实学习问题。