We present SLoMo: a first-of-its-kind framework for transferring skilled motions from casually captured "in the wild" video footage of humans and animals to legged robots. SLoMo works in three stages: 1) synthesize a physically plausible reconstructed key-point trajectory from monocular videos; 2) optimize a dynamically feasible reference trajectory for the robot offline that includes body and foot motion, as well as contact sequences that closely tracks the key points; 3) track the reference trajectory online using a general-purpose model-predictive controller on robot hardware. Traditional motion imitation for legged motor skills often requires expert animators, collaborative demonstrations, and/or expensive motion capture equipment, all of which limits scalability. Instead, SLoMo only relies on easy-to-obtain monocular video footage, readily available in online repositories such as YouTube. It converts videos into motion primitives that can be executed reliably by real-world robots. We demonstrate our approach by transferring the motions of cats, dogs, and humans to example robots including a quadruped (on hardware) and a humanoid (in simulation). To the best knowledge of the authors, this is the first attempt at a general-purpose motion transfer framework that imitates animal and human motions on legged robots directly from casual videos without artificial markers or labels.

翻译：我们提出SLoMo：首个将人类与动物日常动作视频中的技能迁移至四足机器人的通用框架。该系统包含三个阶段：1) 从单目视频中合成具有物理合理性的重构关键点轨迹；2) 通过离线优化生成包含身体与足部运动及接触序列的动力学可行参考轨迹，该轨迹紧密跟随关键点；3) 在机器人硬件上使用通用模型预测控制器在线跟踪参考轨迹。传统四足运动技能模仿通常需要专业动画师、协作演示和/或昂贵的动作捕捉设备，严重限制了可扩展性。而SLoMo仅依赖易于获取的单目视频素材（如YouTube等在线资源），将视频转化为可由实体机器人可靠执行的运动基元。我们通过将猫、狗和人类的动作迁移至四足机器人（硬件实验）和仿人机器人（仿真实验）验证了该方法。据作者所知，这是首个无需人工标记或标签、直接从日常视频中模仿动物和人类动作的通用运动迁移框架。