The full potential of large pretrained models remains largely untapped in control domains like robotics. This is mainly because of the scarcity of data and the computational challenges associated with training or fine-tuning these large models for such applications. Prior work mainly emphasizes effective pretraining of large models for decision-making, with little exploration into how to perform data-efficient continual adaptation of these models for new tasks. Recognizing these constraints, we introduce TAIL (Task-specific Adapters for Imitation Learning), a framework for efficient adaptation to new control tasks. Inspired by recent advancements in parameter-efficient fine-tuning in language domains, we explore efficient fine-tuning techniques -- e.g., Bottleneck Adapters, P-Tuning, and Low-Rank Adaptation (LoRA) -- in TAIL to adapt large pretrained models for new tasks with limited demonstration data. Our extensive experiments in large-scale language-conditioned manipulation tasks comparing prevalent parameter-efficient fine-tuning techniques and adaptation baselines suggest that TAIL with LoRA can achieve the best post-adaptation performance with only 1\% of the trainable parameters of full fine-tuning, while avoiding catastrophic forgetting and preserving adaptation plasticity in continual learning settings.

翻译：大型预训练模型在机器人等控制领域的潜力尚未得到充分挖掘，主要原因是数据稀缺以及针对此类应用训练或微调这些大模型所面临的计算挑战。现有工作主要侧重于为决策任务高效预训练大模型，但鲜有探索如何以数据高效的方式对模型进行持续适应以完成新任务。基于上述局限性，我们提出了TAIL（任务特定模仿学习适配器）框架，旨在高效适应新控制任务。受语言领域参数高效微调技术最新进展的启发，我们在TAIL中探索了高效微调方法（如瓶颈适配器、P-Tuning和低秩适应LoRA），以利用少量演示数据将大模型适配至新任务。我们在大规模语言条件操作任务上进行了广泛实验，对比了主流参数高效微调技术与适应基线方法，结果表明采用LoRA的TAIL仅需全量微调1%的可训练参数即可达到最优适应后性能，同时避免灾难性遗忘并保持持续学习场景中的适应可塑性。