Non-parametric episodic memory can be used to quickly latch onto high-rewarded experience in reinforcement learning tasks. In contrast to parametric deep reinforcement learning approaches in which reward signals need to be back-propagated slowly, these methods only need to discover the solution once, and may then repeatedly solve the task. However, episodic control solutions are stored in discrete tables, and this approach has so far only been applied to discrete action space problems. Therefore, this paper introduces Continuous Episodic Control (CEC), a novel non-parametric episodic memory algorithm for sequential decision making in problems with a continuous action space. Results on several sparse-reward continuous control environments show that our proposed method learns faster than state-of-the-art model-free RL and memory-augmented RL algorithms, while maintaining good long-run performance as well. In short, CEC can be a fast approach for learning in continuous control tasks.

翻译：非参数化情节记忆可快速捕获强化学习任务中的高奖励经验。与参数化深度强化学习方法中需要缓慢反向传播奖励信号不同，此类方法只需发现一次解决方案，即可重复完成任务。然而，情节控制解决方案存储于离散表格中，该方法此前仅适用于离散动作空间问题。为此，本文提出连续情节控制（CEC）——一种面向连续动作空间序贯决策问题的新型非参数化情节记忆算法。在多个稀疏奖励连续控制环境上的实验结果表明：所提方法不仅学习速度优于最先进的无模型强化学习和记忆增强强化学习算法，同时保持了良好的长期性能。简言之，CEC可成为连续控制任务中快速学习的有效途径。