Understanding the emergence of cooperation in systems of computational agents is crucial for the development of effective cooperative AI. Interaction among individuals in real-world settings are often sparse and occur within a broad spectrum of incentives, which often are only partially known. In this work, we explore how cooperation can arise among reinforcement learning agents in scenarios characterised by infrequent encounters, and where agents face uncertainty about the alignment of their incentives with those of others. To do so, we train the agents under a wide spectrum of environments ranging from fully competitive, to fully cooperative, to mixed-motives. Under this type of uncertainty we study the effects of mechanisms, such as reputation and intrinsic rewards, that have been proposed in the literature to foster cooperation in mixed-motives environments. Our findings show that uncertainty substantially lowers the agents' ability to engage in cooperative behaviour, when that would be the best course of action. In this scenario, the use of effective reputation mechanisms and intrinsic rewards boosts the agents' capability to act nearly-optimally in cooperative environments, while greatly enhancing cooperation in mixed-motive environments as well.

翻译：理解计算代理系统中合作的涌现对于开发有效的合作型人工智能至关重要。现实世界中个体之间的交互通常稀疏且发生在广泛的激励范围内，而这些激励往往仅为部分已知。在本研究中，我们探讨在遭遇频率低且代理对其与他人的激励对齐存在不确定性的场景下，强化学习代理如何实现合作。为此，我们在从完全竞争、完全合作到混合动机的广泛环境谱系中训练代理。在这种不确定性下，我们研究了文献中提出的旨在促进混合动机环境中合作的机制（如声誉和内在奖励）的效果。我们的发现表明，当合作本应是最佳行动方案时，不确定性显著降低了代理参与合作行为的能力。在此情境下，有效的声誉机制和内在奖励的使用不仅极大增强了代理在合作环境中的近最优行为能力，同时也显著提升了混合动机环境中的合作水平。