We present CEMA: Causal Explanations for Multi-Agent decision-making; a system to generate causal explanations for agents' decisions in stochastic sequential multi-agent environments. The core of CEMA is a novel causal selection method which, unlike prior work that assumes a specific causal structure, is applicable whenever a probabilistic model for predicting future states of the environment is available. We sample counterfactual worlds with this model which are used to identify and rank the salient causes behind decisions. We also designed CEMA to meet the requirements of social explainable AI. It can generate contrastive explanations based on selected causes and it works as an interaction loop with users to assure relevance and intelligibility for them. We implement CEMA for motion planning for autonomous driving and test it in four diverse simulated scenarios. We show that CEMA correctly and robustly identifies the relevant causes behind decisions and delivers relevant explanations to users' queries.

翻译：我们提出CEMA：一种面向多智能体决策的因果解释系统，用于在随机序列多智能体环境中生成关于智能体决策的因果解释。CEMA的核心是一种新颖的因果选择方法，与先前假设特定因果结构的工作不同，该方法适用于任何能够获取预测环境未来状态的概率模型的情境。我们利用该模型对反事实世界进行采样，以识别并排序决策背后的显著原因。同时，CEMA的设计符合社会可解释人工智能的要求：它能基于选定的原因生成对比性解释，并通过与用户迭代交互确保解释的相关性和可理解性。我们将CEMA应用于自动驾驶的运动规划，并在四种不同的仿真场景中进行了测试。实验表明，CEMA能正确且稳健地识别决策背后的相关原因，并为用户查询提供针对性的解释。