While achieving tremendous success in various fields, existing multi-agent reinforcement learning (MARL) with a black-box neural network architecture makes decisions in an opaque manner that hinders humans from understanding the learned knowledge and how input observations influence decisions. Instead, existing interpretable approaches, such as traditional linear models and decision trees, usually suffer from weak expressivity and low accuracy. To address this apparent dichotomy between performance and interpretability, our solution, MIXing Recurrent soft decision Trees (MIXRTs), is a novel interpretable architecture that can represent explicit decision processes via the root-to-leaf path and reflect each agent's contribution to the team. Specifically, we construct a novel soft decision tree to address partial observability by leveraging the advances in recurrent neural networks, and demonstrate which features influence the decision-making process through the tree-based model. Then, based on the value decomposition framework, we linearly assign credit to each agent by explicitly mixing individual action values to estimate the joint action value using only local observations, providing new insights into how agents cooperate to accomplish the task. Theoretical analysis shows that MIXRTs guarantees the structural constraint on additivity and monotonicity in the factorization of joint action values. Evaluations on the challenging Spread and StarCraft II tasks show that MIXRTs achieves competitive performance compared to widely investigated methods and delivers more straightforward explanations of the decision processes. We explore a promising path toward developing learning algorithms with both high performance and interpretability, potentially shedding light on new interpretable paradigms for MARL.

翻译：尽管现有采用黑盒神经网络架构的多智能体强化学习（MARL）在各领域取得了巨大成功，但其决策过程不透明，阻碍了人类理解所学知识以及输入观测如何影响决策。相比之下，现有可解释方法（如传统线性模型和决策树）通常存在表达能力弱和准确率低的问题。为解决性能与可解释性之间的显著矛盾，我们提出的解决方案——混合递归软决策树（MIXRTs）——是一种新颖的可解释架构，能通过从根节点到叶节点的路径表征显式决策过程，并反映每个智能体对团队的贡献。具体而言，我们构建了一种新型软决策树，通过利用递归神经网络的最新进展解决部分可观测性问题，并通过树状模型展示哪些特征影响决策过程。随后，基于价值分解框架，我们通过仅使用局部观测混合单个动作值来估计联合动作值，从而线性地将信用分配给每个智能体，为理解智能体如何协作完成任务提供了新视角。理论分析表明，MIXRTs在联合动作值分解中保证了可加性和单调性的结构约束。在具有挑战性的Spread和StarCraft II任务上的评估显示，与广泛研究的方法相比，MIXRTs取得了具有竞争力的性能，并提供了更直观的决策过程解释。我们探索了一条开发兼具高性能与可解释性学习算法的有前景路径，有望为MARL提供新的可解释范式。