We present a simulation-based approach for solution of mean field games (MFGs), using the framework of empirical game-theoretical analysis (EGTA). Our primary method employs a version of the double oracle, iteratively adding strategies based on best response to the equilibrium of the empirical MFG among strategies considered so far. We present Fictitious Play (FP) and Replicator Dynamics as two subroutines for computing the empirical game equilibrium. Each subroutine is implemented with a query-based method rather than maintaining an explicit payoff matrix as in typical EGTA methods due to a representation issue we highlight for MFGs. By introducing game model learning and regularization, we significantly improve the sample efficiency of the primary method without sacrificing the overall learning performance. Theoretically, we prove that a Nash equilibrium (NE) exists in the empirical MFG and show the convergence of iterative EGTA to NE of the full MFG with either subroutine. We test the performance of iterative EGTA in various games and show that it outperforms directly applying FP to MFGs in terms of iterations of strategy introduction.

翻译：我们提出了一种基于仿真的方法，用于求解平均场博弈（MFGs），该方法采用了经验博弈论分析（EGTA）框架。我们的主要方法采用双Oracle的变体，基于对当前已考虑策略集合中经验MFG均衡的最优反应迭代式地添加策略。我们引入虚拟博弈（FP）和复制动力学作为两种子程序，用于计算经验博弈均衡。由于我们在MFGs中指出的一项表示问题，每种子程序均采用基于查询的方法实现，而非像典型EGTA方法那样维护显式收益矩阵。通过引入博弈模型学习与正则化，我们在不牺牲整体学习性能的前提下显著提高了主要方法的样本效率。理论上，我们证明了经验MFG中纳什均衡（NE）的存在性，并展示了无论使用哪种子程序，迭代EGTA均能收敛到完整MFG的NE。我们在多种博弈中测试了迭代EGTA的性能，结果表明，在策略引入轮次方面，其优于直接将FP应用于MFGs的方法。