We introduce GEOTACT, a robotic manipulation method capable of retrieving objects buried in granular media. This is a challenging task due to the need to interact with granular media, and doing so based exclusively on tactile feedback, since a buried object can be completely hidden from vision. Tactile feedback is in itself challenging in this context, due to ubiquitous contact with the surrounding media, and the inherent noise level induced by the tactile readings. To address these challenges, we use a learning method trained end-to-end with simulated sensor noise. We show that our problem formulation leads to the natural emergence of learned pushing behaviors that the manipulator uses to reduce uncertainty and funnel the object to a stable grasp despite spurious and noisy tactile readings. We also introduce a training curriculum that enables learning these behaviors in simulation, followed by zero-shot transfer to real hardware. To the best of our knowledge, GEOTACT is the first method to reliably retrieve a number of different objects from a granular environment, doing so on real hardware and with integrated tactile sensing. Videos and additional information can be found at https://jxu.ai/geotact.

翻译：我们提出了一种名为GEOTACT的机器人操控方法，能够从颗粒介质中检索被掩埋的物体。由于需要与颗粒介质交互，且完全依赖触觉反馈（被掩埋物体完全不可见），这是一项具有挑战性的任务。在此场景下，触觉反馈本身也面临挑战——传感器不仅会与周围介质发生持续接触，还会产生固有的高噪声水平。为应对这些挑战，我们采用端到端学习方法，在训练过程中加入模拟传感器噪声。实验表明，我们的问题公式化方法自然催生了学习型推挤行为——机械臂正是通过这种策略，在虚假噪声触觉信号干扰下仍能降低不确定性，最终将物体稳定抓取至目标区域。我们还设计了训练课程体系，使这些行为策略在仿真环境中习得后，能通过零样本迁移直接应用于真实硬件。据我们所知，GEOTACT是首个能在真实硬件上集成触觉感知，从颗粒介质中可靠检索多种物体的方法。补充视频及信息请访问：https://jxu.ai/geotact。