Mean Field Games (MFGs) provide a principled framework for modeling interactions in large population models: at scale, population dynamics become deterministic, with uncertainty entering only through aggregate shocks, or common noise. However, algorithmic progress has been limited since model-free methods are too high variance and exact methods scale poorly. Recent Hybrid Structural Methods (HSMs) use Monte Carlo rollouts for the common noise in combination with exact estimation of the expected return, conditioned on those samples. However, HSMs have not been scaled to Partially Observable settings. We propose Recurrent Structural Policy Gradient (RSPG), the first history-aware HSM for settings involving public information. We also introduce MFAX, our JAX-based framework for MFGs. By leveraging known transition dynamics, RSPG achieves state-of-the-art performance as well as an order-of-magnitude faster convergence and solves, for the first time, a macroeconomics MFG with heterogeneous agents, common noise and history-aware policies. MFAX is publicly available at: https://github.com/CWibault/mfax.

翻译：平均场博弈（MFGs）为大规模群体模型中的交互建模提供了一个原则性框架：在宏观尺度上，群体动态呈现确定性特征，不确定性仅通过聚合冲击或公共噪声引入。然而，由于无模型方法方差过高而精确方法扩展性不足，该领域的算法进展一直受限。近期提出的混合结构方法（HSMs）通过蒙特卡洛推演处理公共噪声，并结合对条件期望收益的精确估计。然而，HSMs尚未扩展至部分可观测场景。本文提出循环结构策略梯度（RSPG），这是首个适用于涉及公共信息场景的历史感知型HSM。同时，我们介绍了基于JAX开发的MFG框架MFAX。通过利用已知的转移动态，RSPG在实现最先进性能的同时，获得了数量级级的收敛加速，并首次求解了包含异质智能体、公共噪声和历史感知策略的宏观经济学MFG问题。MFAX已在https://github.com/CWibault/mfax公开提供。