In recent years, increasing attention has been directed to leveraging pre-trained vision models for motor control. While existing works mainly emphasize the importance of this pre-training phase, the arguably equally important role played by downstream policy learning during control-specific fine-tuning is often neglected. It thus remains unclear if pre-trained vision models are consistent in their effectiveness under different control policies. To bridge this gap in understanding, we conduct a comprehensive study on 14 pre-trained vision models using 3 distinct classes of policy learning methods, including reinforcement learning (RL), imitation learning through behavior cloning (BC), and imitation learning with a visual reward function (VRF). Our study yields a series of intriguing results, including the discovery that the effectiveness of pre-training is highly dependent on the choice of the downstream policy learning algorithm. We show that conventionally accepted evaluation based on RL methods is highly variable and therefore unreliable, and further advocate for using more robust methods like VRF and BC. To facilitate more universal evaluations of pre-trained models and their policy learning methods in the future, we also release a benchmark of 21 tasks across 3 different environments alongside our work.

翻译：近年来，利用预训练视觉模型进行运动控制的研究日益受到关注。尽管现有工作主要强调预训练阶段的重要性，但下游控制策略微调过程中同等关键的策略学习方法却常被忽视。因此，预训练视觉模型在不同控制策略下的有效性是否一致仍不明确。为填补这一认知空白，我们针对14种预训练视觉模型开展了系统性研究，采用三种不同类别的策略学习方法：强化学习（RL）、基于行为克隆（BC）的模仿学习以及基于视觉奖励函数（VRF）的模仿学习。研究结果揭示了若干有趣现象，包括发现预训练有效性高度依赖于下游策略学习算法的选择。我们证明基于RL方法的传统评估基准具有高度波动性且不可靠，并进一步建议采用VRF和BC等更稳健的方法。为促进未来对预训练模型及其策略学习方法进行更通用的评估，我们同时发布了覆盖3种环境中21个任务的基准测试集。