Blind grasping with a dexterous hand is a crucial manipulation capability. Nevertheless, learning such tactile-only policies for real robots remains challenging due to the tactile sim-to-real gap and the limited expressiveness of sparse tactile signals. To bridge this gap, we propose a framework for tactile-only blind grasping that is deployable on a physical multi-fingered robotic hand. Our approach combines three key components. First, we introduce a Real2Sim tactile calibration pipeline that constructs a contact-calibrated digital-twin simulator capable of reproducing real tactile signals. Second, we improve the expressiveness of sparse tactile observations using a layout-aware tactile encoder, which incorporates sensor-geometry priors through self-supervised pretraining. Third, to improve generalization to unseen objects, we train object-specific reinforcement-learning experts in the calibrated simulator and aggregate their successful grasp trajectories into a tactile-conditioned Diffusion Policy. We evaluate our method on a physical LEAP Hand equipped with distributed tactile sensing across 10 seen and 10 unseen objects. The deployed policy achieves a 27\% real-world grasp success rate across all 20 objects, without real-world grasping demonstrations or visual input. Simulation ablations show that layout-aware tactile pretraining improves grasping performance, while sensing-level evaluations confirm that Real2Sim calibration increases the consistency of tactile contact events between simulation and hardware. Together, these results suggest that contact-event calibration, geometry-aware tactile representation learning, and diffusion-based policy aggregation provide an effective path toward tactile-only blind grasping on real dexterous robotic hands. Project page:Dex-Blind-Grasp.github.io.
翻译:灵巧手的盲操作抓取是一项关键的操控能力。然而,由于触觉的仿真到真实差距以及稀疏触觉信号有限的表达能力,在真实机器人上学习仅依赖触觉的策略仍然具有挑战性。为弥合这一差距,我们提出了一个仅依赖触觉的盲操作抓取框架,该框架可在物理多指机器人手上部署。我们的方法结合了三个关键组件。首先,我们引入了一个真实到仿真的触觉标定流水线,该流水线构建了一个能够复现真实触觉信号的接触标定数字孪生仿真器。其次,我们使用一种布局感知的触觉编码器改进稀疏触觉观测的表达能力,该编码器通过自监督预训练融入了传感器几何先验知识。第三,为提升对未见物体的泛化能力,我们在标定后的仿真器中训练了物体特定的强化学习专家,并将其成功的抓取轨迹聚合成一个触觉条件扩散策略。我们在配备分布式触觉传感的物理LEAP手上,对10个见过物体和10个未见物体进行了评估。部署的策略在所有20个物体上实现了27%的真实世界抓取成功率,无需真实世界抓取示范或视觉输入。仿真消融实验表明,布局感知的触觉预训练能提升抓取性能,而传感层面的评估证实,真实到仿真标定提高了仿真与硬件之间触觉接触事件的一致性。这些结果共同表明,接触事件标定、几何感知的触觉表征学习以及基于扩散的策略聚合,为在真实灵巧机器人手上实现仅依赖触觉的盲操作抓取提供了一条有效路径。项目页面:Dex-Blind-Grasp.github.io。