Neuro-symbolic reinforcement learning (NS-RL) has emerged as a promising paradigm for explainable decision-making, characterized by the interpretability of symbolic policies. NS-RL entails structured state representations for tasks with visual observations, but previous methods are unable to refine the structured states with rewards due to a lack of efficiency. Accessibility also remains to be an issue, as extensive domain knowledge is required to interpret symbolic policies. In this paper, we present a framework for learning structured states and symbolic policies jointly, whose key idea is to distill vision foundation models into a scalable perception module and refine it during policy learning. Moreover, we design a pipeline to generate language explanations for policies and decisions using large language models. In experiments on nine Atari tasks, we verify the efficacy of our approach, and we also present explanations for policies and decisions.

翻译：神经符号强化学习（NS-RL）已成为可解释决策领域的一个前景广阔的范式，其特点在于符号策略的可解释性。对于具有视觉观测的任务，NS-RL需要结构化的状态表示，但以往方法由于效率不足而无法利用奖励信号优化结构化状态。可访问性仍然是一个问题，因为解释符号策略需要广泛的领域知识。本文提出一个联合学习结构化状态与符号策略的框架，其核心思想是将视觉基础模型蒸馏为可扩展的感知模块，并在策略学习过程中对其进行优化。此外，我们设计了一个利用大语言模型为策略与决策生成语言解释的流程。在九项Atari游戏任务的实验中，我们验证了该方法的有效性，并展示了策略与决策的可解释性分析。