Multi-agent systems often operate in dynamic and uncertain environments, where agents must not only pursue individual goals but also safeguard collective functionality. This challenge is especially acute in mixed-motive multi-agent systems. This work focuses on cooperative resilience, the ability of agents to anticipate, resist, recover, and transform in the face of disruptions, a critical yet underexplored property in Multi-Agent Reinforcement Learning. We study how reward function design influences resilience in mixed-motive settings and introduce a novel framework that learns reward functions from ranked trajectories, guided by a cooperative resilience metric. Agents are trained in a suite of social dilemma environments using three reward strategies: i) traditional individual reward; ii) resilience-inferred reward; and iii) hybrid that balance both. We explore three reward parameterizations-linear models, hand-crafted features, and neural networks, and employ two preference-based learning algorithms to infer rewards from behavioral rankings. Our results demonstrate that hybrid strategy significantly improve robustness under disruptions without degrading task performance and reduce catastrophic outcomes like resource overuse. These findings underscore the importance of reward design in fostering resilient cooperation, and represent a step toward developing robust multi-agent systems capable of sustaining cooperation in uncertain environments.

翻译：多智能体系统常在动态不确定环境中运行，智能体不仅需追求个体目标，还需保障集体功能。这一挑战在混合动机多智能体系统中尤为突出。本文聚焦合作韧性——即智能体在面对干扰时预测、抵抗、恢复与转型的能力，这是多智能体强化学习中至关重要却尚未充分探索的特性。我们研究奖励函数设计如何影响混合动机场景下的韧性，并提出一种新颖框架：通过合作韧性指标引导，从排序轨迹中学习奖励函数。我们在社会困境环境套件中采用三种奖励策略训练智能体：i) 传统个体奖励；ii) 韧性推断奖励；iii) 平衡二者的混合策略。我们探索了三种奖励参数化方法——线性模型、手工特征和神经网络，并采用两种基于偏好的学习算法从行为排序中推断奖励。实验结果表明，混合策略能在不降低任务性能的前提下显著提升系统在干扰下的鲁棒性，同时减少资源过度使用等灾难性后果。这些发现凸显了奖励设计对培养韧性合作的重要性，为开发能在不确定环境中维持合作的鲁棒多智能体系统迈出了关键一步。