Large behavior models have shown strong dexterous manipulation capabilities by extending imitation learning to large-scale training on multi-task robot data, yet their generalization remains limited by the insufficient robot data coverage. To expand this coverage without costly additional data collection, recent work relies on co-training: jointly learning from target robot data and heterogeneous data modalities. However, how different co-training data modalities and strategies affect policy performance remains poorly understood. We present a large-scale empirical study examining five co-training data modalities: standard vision-language data, dense language annotations for robot trajectories, cross-embodiment robot data, human videos, and discrete robot action tokens across single- and multi-phase training strategies. Our study leverages 4,000 hours of robot and human manipulation data and 50M vision-language samples to train vision-language-action policies. We evaluate 89 policies over 58,000 simulation rollouts and 2,835 real-world rollouts. Our results show that co-training with forms of vision-language and cross-embodiment robot data substantially improves generalization to distribution shifts, unseen tasks, and language following, while discrete action token variants yield no significant benefits. Combining effective modalities produces cumulative gains and enables rapid adaptation to unseen long-horizon dexterous tasks via fine-tuning. Training exclusively on robot data degrades the visiolinguistic understanding of the vision-language model backbone, while co-training with effective modalities restores these capabilities. Explicitly conditioning action generation on chain-of-thought traces learned from co-training data does not improve performance in our simulation benchmark. Together, these results provide practical guidance for building scalable generalist robot policies.

翻译：大规模行为模型通过将模仿学习扩展到多任务机器人数据的大规模训练，已展现出强大的灵巧操作能力，但其泛化能力仍受限于机器人数据覆盖不足。为在不进行昂贵额外数据收集的情况下扩展覆盖范围，近期研究依赖于协同训练：联合学习目标机器人数据与异构数据模态。然而，不同协同训练数据模态及策略如何影响策略性能仍缺乏深入理解。我们开展了一项大规模实证研究，考察了五种协同训练数据模态：标准视觉-语言数据、机器人轨迹的密集语言标注、跨具身机器人数据、人类视频以及离散机器人动作标记，并涵盖单阶段与多阶段训练策略。本研究利用4,000小时的机器人及人类操作数据与5000万视觉-语言样本，训练视觉-语言-动作策略。我们通过58,000次仿真推演与2,835次真实世界推演评估了89种策略。结果表明，采用视觉-语言数据形式及跨具身机器人数据进行协同训练能显著提升对分布偏移、未见任务及语言指令跟随的泛化能力，而离散动作标记变体未带来显著收益。组合有效模态可产生累积增益，并通过微调实现对新增长时域灵巧任务的快速适应。仅使用机器人数据训练会削弱视觉-语言模型骨干的视觉语言理解能力，而结合有效模态的协同训练可恢复这些能力。在我们的仿真基准测试中，显式地基于从协同训练数据中学得的思维链轨迹来条件化动作生成并未提升性能。综上，这些结果为构建可扩展的通用机器人策略提供了实用指导。