Reinforcement learning (RL) is applied in a wide variety of fields. RL enables agents to learn tasks autonomously by interacting with the environment. The more critical the tasks are, the higher the demand for the robustness of the RL systems. Causal RL combines RL and causal inference to make RL more robust. Causal RL agents use a causal representation to capture the invariant causal mechanisms that can be transferred from one task to another. Currently, there is limited research in Causal RL, and existing solutions are usually not complete or feasible for real-world applications. In this work, we propose CausalCF, the first complete Causal RL solution incorporating ideas from Causal Curiosity and CoPhy. Causal Curiosity provides an approach for using interventions, and CoPhy is modified to enable the RL agent to perform counterfactuals. We apply CausalCF to complex robotic tasks and show that it improves the RL agent's robustness using a realistic simulation environment called CausalWorld.

翻译：强化学习（RL）广泛应用于众多领域。它使得智能体能够通过与环境的交互自主地学习任务。任务越关键，对RL系统鲁棒性的要求就越高。因果RL结合了强化学习与因果推断，旨在提升RL的鲁棒性。因果RL智能体利用因果表示来捕捉那些可从一个任务迁移至另一任务的因果不变机制。目前，关于因果RL的研究有限，且现有解决方案在实际应用中往往并不完整或不可行。在本工作中，我们提出了CausalCF——首个完整的因果RL解决方案，它融合了Causal Curiosity和CoPhy的思想。其中，Causal Curiosity提供了使用干预的方法，而CoPhy经过改进使得RL智能体能够执行反事实推理。我们将CausalCF应用于复杂的机器人任务，并利用名为CausalWorld的真实模拟环境证明，它提升了RL智能体的鲁棒性。