In this paper we study the problem of learning multi-step dynamics prediction models (jumpy models) from unlabeled experience and their utility for fast inference of (high-level) plans in downstream tasks. In particular we propose to learn a jumpy model alongside a skill embedding space offline, from previously collected experience for which no labels or reward annotations are required. We then investigate several options of harnessing those learned components in combination with model-based planning or model-free reinforcement learning (RL) to speed up learning on downstream tasks. We conduct a set of experiments in the RGB-stacking environment, showing that planning with the learned skills and the associated model can enable zero-shot generalization to new tasks, and can further speed up training of policies via reinforcement learning. These experiments demonstrate that jumpy models which incorporate temporal abstraction can facilitate planning in long-horizon tasks in which standard dynamics models fail.

翻译：本文研究了从无标签经验中学习多步动力学预测模型（跳跃式模型）的问题，以及这类模型在下游任务中快速推理（高层）规划的应用价值。我们提出在无需标签或奖励标注的情况下，基于先前收集的经验数据，离线联合学习跳跃式模型与技能嵌入空间。随后探讨了多种将所学模块与基于模型的规划或无模型强化学习相结合以加速下游任务学习的方案。在RGB-stacking环境中进行的一系列实验表明：利用所学技能及关联模型进行规划，能够在新任务上实现零样本泛化，并进一步通过强化学习加速策略训练。这些实验证明，融合时间抽象机制的跳跃式模型可有效促进标准动力学模型难以处理的长时域任务规划。