Visuomotor policies trained on human expert demonstrations have recently shown strong performance across a wide range of robotic manipulation tasks. However, these policies remain highly sensitive to domain shifts stemming from background or robot embodiment changes, which limits their generalization capabilities. In this paper, we present ARRO, a novel visual representation that leverages zero-shot open-vocabulary segmentation and object detection models to efficiently mask out task-irrelevant regions of the scene in real time without requiring additional training, modeling of the setup, or camera calibration. By filtering visual distractors and overlaying virtual guides during both training and inference, ARRO improves robustness to scene variations and reduces the need for additional data collection. We extensively evaluate ARRO with Diffusion Policy on a range of tabletop manipulation tasks in both simulation and real-world environments, and further demonstrate its compatibility and effectiveness with generalist robot policies, such as Octo, OpenVLA and Pi Zero. Across all settings in our evaluation, ARRO yields consistent performance gains, allows for selective masking to choose between different objects, and shows robustness even to challenging segmentation conditions. Videos showcasing our results are available at: https://augmented-reality-for-robots.github.io/

翻译：基于人类专家演示训练的视觉运动策略最近在广泛的机器人操作任务中展现出强大的性能。然而，这些策略对由背景或机器人本体变化引起的领域偏移仍然高度敏感，这限制了其泛化能力。本文提出ARRO，一种新颖的视觉表示方法，它利用零样本开放词汇分割和物体检测模型，无需额外训练、场景建模或相机标定，即可实时高效地屏蔽场景中与任务无关的区域。通过在训练和推理阶段过滤视觉干扰物并叠加虚拟引导，ARRO增强了对场景变化的鲁棒性，并减少了对额外数据收集的需求。我们在模拟和真实环境中的一系列桌面操作任务上，使用Diffusion Policy对ARRO进行了广泛评估，并进一步证明了其与通用机器人策略（如Octo、OpenVLA和Pi Zero）的兼容性和有效性。在我们评估的所有设置中，ARRO均带来了一致的性能提升，允许通过选择性掩码在不同物体间进行选择，并即使在具有挑战性的分割条件下也表现出鲁棒性。展示我们结果的视频可在以下网址获取：https://augmented-reality-for-robots.github.io/