In recent years, domains such as natural language processing and image recognition have popularized the paradigm of using large datasets to pretrain representations that can be effectively transferred to downstream tasks. In this work we evaluate how such a paradigm should be done in imitation learning, where both pretraining and finetuning data are trajectories collected by experts interacting with an unknown environment. Namely, we consider a setting where the pretraining corpus consists of multitask demonstrations and the task for each demonstration is set by an unobserved latent context variable. The goal is to use the pretraining corpus to learn a low dimensional representation of the high dimensional (e.g., visual) observation space which can be transferred to a novel context for finetuning on a limited dataset of demonstrations. Among a variety of possible pretraining objectives, we argue that inverse dynamics modeling -- i.e., predicting an action given the observations appearing before and after it in the demonstration -- is well-suited to this setting. We provide empirical evidence of this claim through evaluations on a variety of simulated visuomotor manipulation problems. While previous work has attempted various theoretical explanations regarding the benefit of inverse dynamics modeling, we find that these arguments are insufficient to explain the empirical advantages often observed in our settings, and so we derive a novel analysis using a simple but general environment model.

翻译：近年来，自然语言处理和图像识别等领域普及了一种范式：利用大规模数据集预训练表征，并有效迁移至下游任务。本研究评估了该范式在模仿学习中的实现方式，其中预训练和微调数据均由专家在与未知环境交互过程中收集的轨迹构成。具体而言，我们考虑以下场景：预训练语料库由多任务演示组成，且每个演示的任务由未观测到的潜在上下文变量设定。目标是通过预训练语料库学习高维（如视觉）观测空间的低维表征，并将其迁移至新上下文，从而在有限演示数据集上进行微调。在多种可能的预训练目标中，我们认为逆动力学建模（即根据演示中动作前后的观测预测该动作）特别适合此场景。通过在多种模拟视觉运动操作问题上的评估，我们为这一论点提供了经验证据。尽管以往研究尝试从不同理论角度解释逆动力学建模的优势，但我们发现这些论证不足以说明本研究中观察到的经验优势，因此基于一个简单但通用的环境模型推导出新的分析框架。