Cooperative multi-agent learning plays a crucial role for developing effective strategies to achieve individual or shared objectives in multi-agent teams. In real-world settings, agents may face unexpected failures, such as a robot's leg malfunctioning or a teammate's battery running out. These malfunctions decrease the team's ability to accomplish assigned task(s), especially if they occur after the learning algorithms have already converged onto a collaborative strategy. Current leading approaches in Multi-Agent Reinforcement Learning (MARL) often recover slowly -- if at all -- from such malfunctions. To overcome this limitation, we present the Collaborative Adaptation (CA) framework, highlighting its unique capability to operate in both continuous and discrete domains. Our framework enhances the adaptability of agents to unexpected failures by integrating inter-agent relationships into their learning processes, thereby accelerating the recovery from malfunctions. We evaluated our framework's performance through experiments in both discrete and continuous environments. Empirical results reveal that in scenarios involving unforeseen malfunction, although state-of-the-art algorithms often converge on sub-optimal solutions, the proposed CA framework mitigates and recovers more effectively.

翻译：协作多智能体学习对于开发有效策略以实现多智能体团队中的个体或共同目标具有至关重要的作用。在现实场景中，智能体可能面临意外故障，例如机器人腿部发生故障或队友电池耗尽。这些故障会降低团队完成指定任务的能力，尤其是在学习算法已收敛到协作策略后发生故障时。当前多智能体强化学习（MARL）的主流方法从此类故障中恢复缓慢——甚至可能无法恢复。为克服这一局限，我们提出了协作适应（CA）框架，重点阐述了其在连续和离散领域均可运行的独特能力。该框架通过将智能体间关系整合到其学习过程中，增强了智能体对意外故障的适应能力，从而加速了从故障中的恢复。我们通过在离散和连续环境中的实验评估了该框架的性能。实证结果表明，在涉及突发故障的场景中，尽管现有先进算法常收敛于次优解，但所提出的CA框架能更有效地缓解故障影响并实现恢复。