Learning agent behaviors from observational data has shown to improve our understanding of their decision-making processes, advancing our ability to explain their interactions with the environment and other agents. While multiple learning techniques have been proposed in the literature, there is one particular setting that has not been explored yet: multi agent systems where agent identities remain anonymous. For instance, in financial markets labeled data that identifies market participant strategies is typically proprietary, and only the anonymous state-action pairs that result from the interaction of multiple market participants are publicly available. As a result, sequences of agent actions are not observable, restricting the applicability of existing work. In this paper, we propose a Policy Clustering algorithm, called K-SHAP, that learns to group anonymous state-action pairs according to the agent policies. We frame the problem as an Imitation Learning (IL) task, and we learn a world-policy able to mimic all the agent behaviors upon different environmental states. We leverage the world-policy to explain each anonymous observation through an additive feature attribution method called SHAP (SHapley Additive exPlanations). Finally, by clustering the explanations we show that we are able to identify different agent policies and group observations accordingly. We evaluate our approach on simulated synthetic market data and a real-world financial dataset. We show that our proposal significantly and consistently outperforms the existing methods, identifying different agent strategies.

翻译：从观测数据中学习智能体行为已证明能增进对其决策过程的理解，从而提升我们解释其与环境及其他智能体交互的能力。尽管文献中已提出多种学习技术，但仍有一个特定场景尚未被探索：智能体身份保持匿名的多智能体系统。例如，在金融市场中，标识市场参与者策略的标注数据通常具有专有性，仅有来自多个市场参与者交互产生的匿名状态-动作对可供公开获取。因此，智能体动作序列无法被观测，限制了现有工作的适用性。本文提出一种名为K-SHAP的策略聚类算法，该算法能根据智能体策略对匿名状态-动作对进行分组学习。我们将该问题构建为模仿学习（Imitation Learning, IL）任务，并学习一种能够模仿所有智能体在不同环境状态下行为的全局策略。我们利用该全局策略通过一种名为SHAP（SHapley Additive exPlanations）的加性特征归因方法解释每个匿名观测。最后，通过对解释结果进行聚类，我们证明能够识别不同智能体策略并据此对观测进行分组。我们在模拟合成市场数据和真实金融数据集上评估了该方法，结果表明我们的方案显著且一致地优于现有方法，能够成功识别不同智能体策略。