We present a neural network approach to transfer the motion from a single image of an articulated object to a rest-state (i.e., unarticulated) 3D model. Our network learns to predict the object's pose, part segmentation, and corresponding motion parameters to reproduce the articulation shown in the input image. The network is composed of three distinct branches that take a shared joint image-shape embedding and is trained end-to-end. Unlike previous methods, our approach is independent of the topology of the object and can work with objects from arbitrary categories. Our method, trained with only synthetic data, can be used to automatically animate a mesh, infer motion from real images, and transfer articulation to functionally similar but geometrically distinct 3D models at test time.
翻译:我们提出一种神经网络方法,用于将铰接物体单张图像中的运动迁移至静态(即未铰接)三维模型。该网络通过学习预测物体的姿态、部件分割及相应运动参数,从而复现输入图像所呈现的铰接状态。网络由三个独立分支组成,共享联合图像-形状嵌入,并通过端到端方式训练。与以往方法不同,本方法不依赖物体拓扑结构,可处理任意类别的物体。仅使用合成数据训练的模型,能够自动为网格模型添加动画、从真实图像推断运动,并在测试阶段将铰接迁移至功能相似但几何结构不同的三维模型。