Agent-based models (ABMs) have shown promise for modelling various real world phenomena incompatible with traditional equilibrium analysis. However, a critical concern is the manual definition of behavioural rules in ABMs. Recent developments in multi-agent reinforcement learning (MARL) offer a way to address this issue from an optimisation perspective, where agents strive to maximise their utility, eliminating the need for manual rule specification. This learning-focused approach aligns with established economic and financial models through the use of rational utility-maximising agents. However, this representation departs from the fundamental motivation for ABMs: that realistic dynamics emerging from bounded rationality and agent heterogeneity can be modelled. To resolve this apparent disparity between the two approaches, we propose a novel technique for representing heterogeneous processing-constrained agents within a MARL framework. The proposed approach treats agents as constrained optimisers with varying degrees of strategic skills, permitting departure from strict utility maximisation. Behaviour is learnt through repeated simulations with policy gradients to adjust action likelihoods. To allow efficient computation, we use parameterised shared policy learning with distributions of agent skill levels. Shared policy learning avoids the need for agents to learn individual policies yet still enables a spectrum of bounded rational behaviours. We validate our model's effectiveness using real-world data on a range of canonical $n$-agent settings, demonstrating significantly improved predictive capability.

翻译：基于智能体的模型（ABMs）在建模传统均衡分析难以处理的各类现实世界现象方面展现出潜力。然而，一个关键问题在于ABM中行为规则的人为定义。最近多智能体强化学习（MARL）的发展从优化视角提供了应对方案——智能体致力于最大化自身效用，从而消除了手动规则设定的需求。这种以学习为导向的方法通过引入理性效用最大化智能体，与既有的经济金融模型保持了一致性。但这种表征方式背离了ABM的根本动机：即通过建模有限理性和智能体异质性产生的现实动态。为解决这两种方法间的明显分歧，我们提出了一种新颖技术，在MARL框架中表征异质性的处理受限智能体。该方法将智能体视为具有不同程度策略技巧的约束优化者，允许偏离严格的效用最大化原则。通过策略梯度法在重复模拟中调整行为概率来学习行为。为实现高效计算，我们采用参数化共享策略学习结合智能体技能水平的分布。共享策略学习无需每个智能体单独学习策略，仍能支持连续谱系的有限理性行为。我们利用真实世界数据在经典n智能体设置中验证了模型有效性，显著提升了预测能力。