We present a novel approach to address the challenge of generalization in offline reinforcement learning (RL), where the agent learns from a fixed dataset without any additional interaction with the environment. Specifically, we aim to improve the agent's ability to generalize to out-of-distribution goals. To achieve this, we propose to learn a dynamics model and check if it is equivariant with respect to a fixed type of transformation, namely translations in the state space. We then use an entropy regularizer to increase the equivariant set and augment the dataset with the resulting transformed samples. Finally, we learn a new policy offline based on the augmented dataset, with an off-the-shelf offline RL algorithm. Our experimental results demonstrate that our approach can greatly improve the test performance of the policy on the considered environments.

翻译：我们提出了一种新颖方法，旨在解决离线强化学习中智能体仅从固定数据集学习（无法与环境进行额外交互）时的泛化挑战。具体而言，我们致力于提升智能体在分布外目标上的泛化能力。为此，我们首先学习一个动力学模型，并检验其是否关于特定变换类型（即状态空间中的平移变换）具有等变性。随后，我们利用熵正则化器扩大等变集合，并通过生成的变换样本对数据集进行增强。最终，基于增强后的数据集，我们采用现成的离线强化学习算法学习新的策略。实验结果表明，该方法在所测试环境中显著提升了策略的测试性能。