Dexterous robotic manipulator teleoperation is widely used in many applications, either where it is convenient to keep the human inside the control loop, or to train advanced robot agents. So far, this technology has been used in combination with camera systems with remarkable success. On the other hand, only a limited number of studies have focused on leveraging haptic feedback from tactile sensors in contexts where camera-based systems fail, such as due to self-occlusions or poor light conditions like smoke. This study demonstrates the feasibility of precise pick-and-place teleoperation without cameras by leveraging tactile-based 3D object reconstruction in VR and providing haptic feedback to a blindfolded user. Our preliminary results show that integrating these technologies enables the successful completion of telemanipulation tasks previously dependent on cameras, paving the way for more complex future applications.

翻译：灵巧机器人遥操作在许多应用中得到了广泛使用，无论是出于将人类保持在控制回路中的便利性，还是为了训练高级机器人智能体。迄今为止，该技术已与视觉系统结合使用，并取得了显著成功。另一方面，仅有有限的研究关注于在视觉系统失效（例如因自遮挡或烟雾等恶劣光照条件）的场景中，利用来自触觉传感器的触觉反馈。本研究通过利用基于触觉的虚拟现实三维物体重建并向蒙眼用户提供触觉反馈，证明了无需视觉即可实现精确抓放遥操作的可行性。我们的初步结果表明，整合这些技术能够成功完成以往依赖视觉的遥操作任务，为未来更复杂的应用铺平了道路。