We present a system for collision-free control of a robot manipulator that uses only RGB views of the world. Perceptual input of a tabletop scene is provided by multiple images of an RGB camera (without depth) that is either handheld or mounted on the robot end effector. A NeRF-like process is used to reconstruct the 3D geometry of the scene, from which the Euclidean full signed distance function (ESDF) is computed. A model predictive control algorithm is then used to control the manipulator to reach a desired pose while avoiding obstacles in the ESDF. We show results on a real dataset collected and annotated in our lab.

翻译：我们提出了一套仅利用世界RGB视角来实现机器人机械臂无碰撞控制的系统。桌面场景的感知输入由多张RGB相机（无深度信息）图像提供，该相机可手持或安装在机械臂末端执行器上。采用类似NeRF的技术流程重建场景的三维几何结构，并从中计算欧几里得全符号距离函数（ESDF）。随后使用模型预测控制算法，使机械臂在规避ESDF中障碍物的同时，达到目标位姿。我们在实验室采集并标注的真实数据集上展示了实验结果。