Deep Q Networks (DQN) have shown remarkable success in various reinforcement learning tasks. However, their reliance on associative learning often leads to the acquisition of spurious correlations, hindering their problem-solving capabilities. In this paper, we introduce a novel approach to integrate causal principles into DQNs, leveraging the PEACE (Probabilistic Easy vAriational Causal Effect) formula for estimating causal effects. By incorporating causal reasoning during training, our proposed framework enhances the DQN's understanding of the underlying causal structure of the environment, thereby mitigating the influence of confounding factors and spurious correlations. We demonstrate that integrating DQNs with causal capabilities significantly enhances their problem-solving capabilities without compromising performance. Experimental results on standard benchmark environments showcase that our approach outperforms conventional DQNs, highlighting the effectiveness of causal reasoning in reinforcement learning. Overall, our work presents a promising avenue for advancing the capabilities of deep reinforcement learning agents through principled causal inference.

翻译：深度Q网络（DQN）在各种强化学习任务中已展现出卓越的成功。然而，其对关联学习的依赖常常导致习得虚假相关性，从而阻碍其问题解决能力。本文提出一种新颖方法，将因果原理整合到DQN中，利用PEACE（概率简易变分因果效应）公式来估计因果效应。通过在训练过程中融入因果推理，我们提出的框架增强了DQN对环境底层因果结构的理解，从而减轻了混杂因素和虚假相关性的影响。我们证明，为DQN整合因果能力能在不牺牲性能的前提下，显著提升其问题解决能力。在标准基准环境上的实验结果表明，我们的方法优于传统DQN，突显了因果推理在强化学习中的有效性。总体而言，我们的工作为通过原则性因果推断来推进深度强化学习智能体的能力，提供了一条前景广阔的路径。