Vision-based teleoperation offers the possibility to endow robots with human-level intelligence to physically interact with the environment, while only requiring low-cost camera sensors. However, current vision-based teleoperation systems are designed and engineered towards a particular robot model and deploy environment, which scales poorly as the pool of the robot models expands and the variety of the operating environment increases. In this paper, we propose AnyTeleop, a unified and general teleoperation system to support multiple different arms, hands, realities, and camera configurations within a single system. Although being designed to provide great flexibility to the choice of simulators and real hardware, our system can still achieve great performance. For real-world experiments, AnyTeleop can outperform a previous system that was designed for a specific robot hardware with a higher success rate, using the same robot. For teleoperation in simulation, AnyTeleop leads to better imitation learning performance, compared with a previous system that is particularly designed for that simulator. Project page: http://anyteleop.com/.

翻译：基于视觉的遥操作系统有望赋予机器人人类级别的智能以物理交互环境，且仅需低成本摄像头传感器。然而，当前基于视觉的遥操作系统是针对特定机器人型号和部署环境专门设计的，随着机器人型号库的扩展和操作环境多样性的增加，其可扩展性较差。本文提出AnyTeleop，一种统一且通用的遥操作系统，可在单一系统中支持多种不同的手臂、手部、现实场景及摄像头配置。尽管设计旨在为模拟器和真实硬件选择提供极大灵活性，该系统仍能实现卓越性能。在真实世界实验中，AnyTeleop以更高成功率超越了此前为特定机器人硬件设计的系统（使用相同机器人）。在仿真遥操作中，与针对该模拟器特别设计的先前系统相比，AnyTeleop可带来更优的模仿学习性能。项目页面：http://anyteleop.com/。