Pretrained vision foundation models (VFMs) advance robotic learning via rich visual representations, yet individual VFMs typically excel only in specific domains, limiting generality across tasks. Distilling multiple VFMs into a unified representation for policy can mitigate this limitation but often yields inflexible task-specific feature selection and requires costly full re-training to incorporate robot-domain knowledge. We propose VER, a Vision Expert transformer for Robot learning. During pretraining, VER distills multiple VFMs into a vision expert library. It then fine-tunes only a lightweight routing network (fewer than 0.4% of parameters) to dynamically select task-relevant experts from the pretrained library for downstream robot tasks. We further introduce Patchwise Expert Routing with Curriculum Top-K Annealing to improve both flexibility and precision of dynamic expert selection. Moreover, VER supports parameter-efficient finetuning for scalable expert utilization and adaptive robot-domain knowledge integration. Across 17 diverse robotic tasks and multiple policy heads, VER achieves state-of-the-art performance. We find that VER reduces large-norm outliers in task-irrelevant regions (e.g., background) and concentrates on task-critical regions. Visualizations and codes can be found in https://yixiaowang7.github.io/ver_page/.

翻译：预训练视觉基础模型通过丰富的视觉表征推动了机器人学习的发展，但单个视觉基础模型通常仅在特定领域表现优异，限制了跨任务泛化能力。将多个视觉基础模型蒸馏为统一表征的策略能缓解这一局限，但往往导致僵化的任务特定特征选择，且引入机器人领域知识需要代价高昂的完整重训练。我们提出VER——面向机器人学习的视觉专家Transformer。预训练阶段，VER将多个视觉基础模型蒸馏至视觉专家库中；随后仅微调轻量级路由网络（参数不足0.4%）即可从预训练专家库中动态选择任务相关专家，用于下游机器人任务。我们进一步引入基于课程式Top-K退火的逐块专家路由，提升动态专家选择的灵活性与精准度。此外，VER支持参数高效微调以实现可扩展的专家利用与自适应的机器人领域知识整合。在涵盖17种不同机器人任务与多种策略头的实验中，VER取得最先进性能。研究发现，VER能减少任务无关区域（如背景）中的大范数离群值，并聚焦于任务关键区域。可视化结果与代码请见https://yixiaowang7.github.io/ver_page/。