Reorienting diverse objects with a multi-fingered hand is a challenging task. Current methods in robotic in-hand manipulation are either object-specific or require permanent supervision of the object state from visual sensors. This is far from human capabilities and from what is needed in real-world applications. In this work, we address this gap by training shape-conditioned agents to reorient diverse objects in hand, relying purely on tactile feedback (via torque and position measurements of the fingers' joints). To achieve this, we propose a learning framework that exploits shape information in a reinforcement learning policy and a learned state estimator. We find that representing 3D shapes by vectors from a fixed set of basis points to the shape's surface, transformed by its predicted 3D pose, is especially helpful for learning dexterous in-hand manipulation. In simulation and real-world experiments, we show the reorientation of many objects with high success rates, on par with state-of-the-art results obtained with specialized single-object agents. Moreover, we show generalization to novel objects, achieving success rates of $\sim$90% even for non-convex shapes.

翻译：多指手重新定向多样化物体是一项具有挑战性的任务。当前机器人手中操作方法要么针对特定物体，要么需要视觉传感器对物体状态进行持续监控。这与人类能力及实际应用需求相去甚远。在本研究中，我们通过训练形状条件智能体来实现多样化物体的手中重定向，仅依赖触觉反馈（通过手指关节的扭矩和位置测量）。为实现这一目标，我们提出了一个学习框架，在强化学习策略和习得状态估计器中利用形状信息。我们发现，通过从固定基础点集到形状表面的向量（经其预测三维姿态变换）来表示三维形状，对学习灵巧手中操作特别有效。在仿真和现实实验中，我们展示了多种物体的重定向具有高成功率，与专用单物体智能体获得的最先进结果相当。此外，我们展示了向新物体的泛化能力，即使对于非凸形状也能达到约90%的成功率。