We propose a physics-based method for synthesizing dexterous hand-object interactions in a full-body setting. While recent advancements have addressed specific facets of human-object interactions, a comprehensive physics-based approach remains a challenge. Existing methods often focus on isolated segments of the interaction process and rely on data-driven techniques that may result in artifacts. In contrast, our proposed method embraces reinforcement learning (RL) and physics simulation to mitigate the limitations of data-driven approaches. Through a hierarchical framework, we first learn skill priors for both body and hand movements in a decoupled setting. The generic skill priors learn to decode a latent skill embedding into the motion of the underlying part. A high-level policy then controls hand-object interactions in these pretrained latent spaces, guided by task objectives of grasping and 3D target trajectory following. It is trained using a novel reward function that combines an adversarial style term with a task reward, encouraging natural motions while fulfilling the task incentives. Our method successfully accomplishes the complete interaction task, from approaching an object to grasping and subsequent manipulation. We compare our approach against kinematics-based baselines and show that it leads to more physically plausible motions.

翻译：我们提出了一种基于物理的方法，用于在全身体态下合成灵巧的手-物体交互。尽管近期进展已解决人类-物体交互的特定方面，但全面的物理驱动方法仍面临挑战。现有方法通常聚焦于交互过程的孤立片段，并依赖可能产生伪影的数据驱动技术。相比之下，我们的方法采用强化学习和物理模拟来减轻数据驱动方法的局限性。通过层级框架，我们首先在解耦设置中学习身体与手部运动的技能先验。通用技能先验学习将潜在技能嵌入解码为对应部位的运动。随后，高层策略在这些预训练的潜在空间中控制手-物体交互，受抓取与三维目标轨迹跟踪的任务目标引导。该策略通过结合对抗性风格项与任务奖励的新型奖励函数进行训练，在完成任务激励的同时促进自然运动。我们的方法成功实现了从接近物体到抓取及后续操作的完整交互任务。通过对比基于运动学的基线方法，我们证明本方法能产生更物理合理的运动。