Value decomposition is widely used in cooperative multi-agent reinforcement learning, however, its implicit credit assignment mechanism is not yet fully understood due to black-box networks. In this work, we study an interpretable value decomposition framework via the family of generalized additive models. We present a novel method, named Neural Attention Additive Q-learning (N$\text{A}^\text{2}$Q), providing inherent intelligibility of collaboration behavior. N$\text{A}^\text{2}$Q can explicitly factorize the optimal joint policy induced by enriching shape functions to model all possible coalitions of agents into individual policies. Moreover, we construct identity semantics to promote estimating credits together with the global state and individual value functions, where local semantic masks help us diagnose whether each agent captures relevant-task information. Extensive experiments show that N$\text{A}^\text{2}$Q consistently achieves superior performance compared to different state-of-the-art methods on all challenging tasks, while yielding human-like interpretability.

翻译：价值分解被广泛用于合作式多智能体强化学习中，然而由于黑箱网络的存在，其隐式信用分配机制尚未被完全理解。本文通过广义加性模型族研究了一种可解释的价值分解框架。我们提出了一种名为神经注意力加性Q学习（N$\text{A}^\text{2}$Q）的新方法，该方法能提供协作行为的内在可理解性。N$\text{A}^\text{2}$Q可通过丰富形状函数显式分解由所有可能智能体联盟诱导的最优联合策略为个体策略。此外，我们构建了身份语义机制，以促进全局状态与个体价值函数共同进行信用估计，其中局部语义掩码可帮助诊断每个智能体是否捕获了相关任务信息。大量实验表明，N$\text{A}^\text{2}$Q在所有具有挑战性的任务中均持续取得了优于不同最先进方法的性能，同时展现出类人的可解释性。