Recent progress in vision language foundation models has shown their ability to understand multimodal data and resolve complicated vision language tasks, including robotics manipulation. We seek a straightforward way of making use of existing vision-language models (VLMs) with simple fine-tuning on robotics data. To this end, we derive a simple and novel vision-language manipulation framework, dubbed RoboFlamingo, built upon the open-source VLMs, OpenFlamingo. Unlike prior works, RoboFlamingo utilizes pre-trained VLMs for single-step vision-language comprehension, models sequential history information with an explicit policy head, and is slightly fine-tuned by imitation learning only on language-conditioned manipulation datasets. Such a decomposition provides RoboFlamingo the flexibility for open-loop control and deployment on low-performance platforms. By exceeding the state-of-the-art performance with a large margin on the tested benchmark, we show RoboFlamingo can be an effective and competitive alternative to adapt VLMs to robot control. Our extensive experimental results also reveal several interesting conclusions regarding the behavior of different pre-trained VLMs on manipulation tasks. We believe RoboFlamingo has the potential to be a cost-effective and easy-to-use solution for robotics manipulation, empowering everyone with the ability to fine-tune their own robotics policy.

翻译：近期视觉语言基础模型的进展表明，它们具备理解多模态数据并解决复杂视觉语言任务（包括机器人操作）的能力。我们探索了一种直接利用现有视觉语言模型（VLM）的方法——仅需对机器人数据进行简单微调。为此，我们提出一个简单新颖的视觉语言操作框架，命名为RoboFlamingo，该框架基于开源VLM模型OpenFlamingo构建。与先前工作不同，RoboFlamingo利用预训练VLM进行单步视觉语言理解，通过显式策略头建模序列历史信息，并仅针对语言条件操作数据集通过模仿学习进行轻微微调。这种分解为RoboFlamingo提供了进行开环控制及在低性能平台上部署的灵活性。通过在测试基准上以显著优势超越当前最优性能，我们证明RoboFlamingo是将VLM适配到机器人控制任务的有效且具有竞争力的方案。广泛的实验结果还揭示了不同预训练VLM在操作任务上行为模式的多项有趣结论。我们认为RoboFlamingo有望成为机器人操作领域兼具成本效益与易用性的解决方案，使每个人都能自主微调专属的机器人策略。