Altruistic cooperation is costly yet socially desirable. As a result, agents struggle to learn cooperative policies through independent reinforcement learning (RL). Indirect reciprocity, where agents consider their interaction partner's reputation, has been shown to stabilise cooperation in homogeneous, idealised populations. However, more realistic settings are comprised of heterogeneous agents with different characteristics and group-based social identities. We study cooperation when agents are stratified into two such groups, and allow reputation updates and actions to depend on group information. We consider two modelling approaches: evolutionary game theory, where we comprehensively search for social norms (i.e., rules to assign reputations) leading to cooperation and fairness; and RL, where we consider how the stochastic dynamics of policy learning affects the analytically identified equilibria. We observe that a defecting majority leads the minority group to defect, but not the inverse. Moreover, changing the norms that judge in and out-group interactions can steer a system towards either fair or unfair cooperation. This is made clearer when moving beyond equilibrium analysis to independent RL agents, where convergence to fair cooperation occurs with a narrower set of norms. Our results highlight that, in heterogeneous populations with reputations, carefully defining interaction norms is fundamental to tackle both dilemmas of cooperation and of fairness.

翻译：利他合作虽成本高昂但具有社会价值。因此，智能体难以通过独立强化学习掌握合作策略。间接互惠机制——即智能体在互动时考虑对方声誉——已被证明能在同质化理想群体中稳定合作关系。然而，更现实的场景往往包含具有不同特征和群体社会身份的异质智能体。本研究探讨当智能体被划分为两个群体，且允许声誉更新和行为决策依赖群体信息时的合作问题。我们采用两种建模方法：进化博弈论中系统搜索能产生合作与公平的社会规范（即声誉分配规则）；强化学习中探究策略学习的随机动态如何影响理论均衡。研究发现：多数群体的背叛行为会导致少数群体效仿，反之则不成立；调整评判群内与群际互动的规范可将系统导向公平或不公平的合作状态。当从均衡分析转向独立强化学习智能体时，这种效应更为明显——仅当规范集更为严格时才会收敛至公平合作。本研究强调：在具有声誉机制的异质群体中，精确定义互动规范是解决合作困境与公平困境的共同基石。