Dexterous hand manipulation increasingly relies on large-scale motion datasets with precise hand-object trajectory data. However, existing resources such as DexYCB and HO3D are primarily optimized for visual alignment but often yield physically implausible interactions when replayed in physics simulators, including penetration, missed contact, and unstable grasps. We propose a simulation-in-the-loop refinement framework that converts these visually aligned trajectories into physically executable ones. Our core contribution is to formulate this as a tractable black-box optimization problem. We parameterize the hand's motion using a low-dimensional, spline-based representation built on sparse temporal keyframes. This allows us to use a powerful gradient-free optimizer, CMA-ES, to treat the high-fidelity physics engine as a black-box objective function. Our method finds motions that simultaneously maximize physical success (e.g., stable grasp and lift) while minimizing deviation from the original human demonstration. Compared to MANIPTRANS-recent transfer pipelines, our approach achieves lower hand and object pose errors during replay and more accurately recovers hand-object physical interactions. Our approach provides a general and scalable method for converting visual demonstrations into physically valid trajectories, enabling the generation of high-fidelity data crucial for robust policy learning.

翻译：灵巧手操作日益依赖于具备精确手-物轨迹数据的大规模运动数据集。然而，现有资源如DexYCB和HO3D主要针对视觉对齐进行优化，但在物理模拟器中回放时往往产生物理上不合理的交互，包括穿透、接触缺失及抓握不稳定等问题。我们提出一种模拟循环优化框架，将这些视觉对齐的轨迹转化为物理上可执行的轨迹。我们的核心贡献在于将此问题表述为一个可处理的**黑盒优化问题**。我们采用基于稀疏时间关键帧的低维样条表示对手部运动进行参数化。这使得我们能够利用强大的无梯度优化器CMA-ES，将高保真物理引擎视为黑盒目标函数。我们的方法能够找到同时最大化物理成功率（例如稳定抓握与提升）并最小化与原始人类演示偏差的运动轨迹。与近期迁移流程MANIPTRANS相比，我们的方法在回放过程中实现了更低的手部与物体姿态误差，并更准确地恢复了手-物物理交互。本方法为将视觉演示转化为物理有效轨迹提供了一种通用且可扩展的解决方案，从而能够生成对鲁棒策略学习至关重要的高保真数据。