We present RoboPanoptes, a capable yet practical robot system that achieves whole-body dexterity through whole-body vision. Its whole-body dexterity allows the robot to utilize its entire body surface for manipulation, such as leveraging multiple contact points or navigating constrained spaces. Meanwhile, whole-body vision uses a camera system distributed over the robot's surface to provide comprehensive, multi-perspective visual feedback of its own and the environment's state. At its core, RoboPanoptes uses a whole-body visuomotor policy that learns complex manipulation skills directly from human demonstrations, efficiently aggregating information from the distributed cameras while maintaining resilience to sensor failures. Together, these design aspects unlock new capabilities and tasks, allowing RoboPanoptes to unbox in narrow spaces, sweep multiple or oversized objects, and succeed in multi-step stowing in cluttered environments, outperforming baselines in adaptability and efficiency. Results are best viewed on https://robopanoptes.github.io.

翻译：我们提出了RoboPanoptes，一个能力强且实用的机器人系统，通过全身视觉实现全身灵巧性。其全身灵巧性使机器人能够利用其整个体表进行操控，例如利用多个接触点或在受限空间中导航。同时，全身视觉利用分布在机器人表面的摄像头系统，提供对其自身状态及环境状态的全面、多视角视觉反馈。RoboPanoptes的核心是一个全身视觉运动策略，该策略直接从人类演示中学习复杂的操控技能，有效聚合来自分布式摄像头的信息，同时保持对传感器故障的鲁棒性。这些设计方面共同解锁了新的能力和任务，使RoboPanoptes能够在狭窄空间中拆箱、清扫多个或超大物体，并在杂乱环境中成功完成多步骤收纳任务，在适应性和效率方面均优于基线方法。完整结果请访问 https://robopanoptes.github.io 查看。