Current macroeconomic models with agent heterogeneity can be broadly divided into two main groups. Heterogeneous-agent general equilibrium (GE) models, such as those based on Heterogeneous Agent New Keynesian (HANK) or Krusell-Smith (KS) approaches, rely on GE and 'rational expectations', somewhat unrealistic assumptions that make the models very computationally cumbersome, which in turn limits the amount of heterogeneity that can be modelled. In contrast, agent-based models (ABMs) can flexibly encompass a large number of arbitrarily heterogeneous agents, but typically require the specification of explicit behavioural rules, which can lead to a lengthy trial-and-error model-development process. To address these limitations, we introduce MARL-BC, a framework that integrates deep multi-agent reinforcement learning (MARL) with real business cycle (RBC) models. We demonstrate that MARL-BC can: (1) recover textbook RBC results when using a single agent; (2) recover the results of the mean-field KS model using a large number of identical agents; and (3) effectively simulate rich heterogeneity among agents, a hard task for traditional GE approaches. Our framework can be thought of as an ABM if used with a variety of heterogeneous interacting agents, and can reproduce GE results in limit cases. As such, it is a step towards a synthesis of these often opposed modelling paradigms.

翻译：当前具有智能体异质性的宏观经济模型主要可分为两大类。基于异质性智能体一般均衡（GE）的模型，例如采用异质性智能体新凯恩斯（HANK）或克鲁塞尔-史密斯（KS）方法的模型，依赖于一般均衡与“理性预期”等部分脱离现实的假设，这些假设使得模型计算极为繁重，从而限制了可建模的异质性程度。相比之下，基于智能体的模型（ABM）能够灵活容纳大量任意异质的智能体，但通常需要设定明确的行为规则，这可能导致冗长的试错式模型开发过程。为克服这些局限，我们提出了MARL-BC框架，该框架将深度多智能体强化学习（MARL）与真实经济周期（RBC）模型相结合。我们证明MARL-BC能够：（1）在单一智能体场景下复现经典RBC理论结果；（2）通过大量同质智能体复现平均场KS模型的结果；（3）有效模拟智能体间丰富的异质性特征——这对传统一般均衡方法而言是艰巨任务。当使用多样化的异质交互智能体时，本框架可视为一种基于智能体的模型，并能在极限情况下重现一般均衡结果。因此，该框架为融合这两种常被对立的建模范式迈出了重要一步。