The Mean-Field approximation is a tractable approach for studying large population dynamics. However, its assumption on homogeneity and universal connections among all agents limits its applicability in many real-world scenarios. Multi-Population Mean-Field Game (MP-MFG) models have been introduced in the literature to address these limitations. When the underlying Stochastic Block Model is known, we show that a Policy Mirror Ascent algorithm finds the MP-MFG Nash Equilibrium. In more realistic scenarios where the block model is unknown, we propose a re-sampling scheme from a graphon integrated with the finite N-player MP-MFG model. We develop a novel learning framework based on a Graphon Game with Re-Sampling (GGR-S) model, which captures the complex network structures of agents' connections. We analyze GGR-S dynamics and establish the convergence to dynamics of MP-MFG. Leveraging this result, we propose an efficient sample-based N-player Reinforcement Learning algorithm for GGR-S without population manipulation, and provide a rigorous convergence analysis with finite sample guarantee.

翻译：平均场近似是研究大规模群体动力学的一种可行方法。然而，其对同质性和所有智能体之间通用连接的假设限制了其在许多现实场景中的适用性。文献中引入了多群体平均场博弈（MP-MFG）模型以解决这些局限性。当底层随机块模型已知时，我们证明策略镜像上升算法能够找到MP-MFG纳什均衡。在块模型未知的更现实场景中，我们提出了一种基于图论与有限N玩家MP-MFG模型集成的重采样方案。我们开发了一种基于重采样图论博弈（GGR-S）模型的新型学习框架，该框架捕捉了智能体连接的复杂网络结构。我们分析了GGR-S动力学，并建立了其收敛到MP-MFG动力学的性质。利用这一结果，我们提出了一种无需群体操控的、基于样本的高效N玩家GGR-S强化学习算法，并提供了具有有限样本保证的严格收敛性分析。