In this work, we aim to learn a unified vision-based policy for a multi-fingered robot hand to manipulate different objects in diverse poses. Though prior work has demonstrated that human videos can benefit policy learning, performance improvement has been limited by physically implausible trajectories extracted from videos. Moreover, reliance on privileged object information such as ground-truth object states further limits the applicability in realistic scenarios. To address these limitations, we propose a new framework ViViDex to improve vision-based policy learning from human videos. It first uses reinforcement learning with trajectory guided rewards to train state-based policies for each video, obtaining both visually natural and physically plausible trajectories from the video. We then rollout successful episodes from state-based policies and train a unified visual policy without using any privileged information. A coordinate transformation method is proposed to significantly boost the performance. We evaluate our method on three dexterous manipulation tasks and demonstrate a large improvement over state-of-the-art algorithms.

翻译：摘要：本研究旨在为多指机械手学习统一的视觉策略，使其能够以不同姿态操控多种物体。尽管先前研究表明人类视频可辅助策略学习，但视频中提取的物理不可行轨迹限制了性能提升。此外，依赖真实物体状态等特权信息进一步制约了该方法在现实场景中的应用。为解决上述问题，我们提出新框架ViViDex以改进基于人类视频的视觉策略学习。首先采用轨迹引导奖励的强化学习为每个视频训练基于状态的策略，从而获得兼具视觉自然性与物理可行性的运动轨迹。随后基于状态策略的演示生成成功轨迹，在不使用任何特权信息条件下训练统一的视觉策略。我们提出坐标变换方法显著提升性能。在三个灵巧操作任务上的评估表明，本方法相较现有最优算法实现了显著性能提升。