This paper introduces a novel framework named D-LORD (Double Latent Optimization for Representation Disentanglement), which is designed for motion stylization (motion style transfer and motion retargeting). The primary objective of this framework is to separate the class and content information from a given motion sequence using a data-driven latent optimization approach. Here, class refers to person-specific style, such as a particular emotion or an individual's identity, while content relates to the style-agnostic aspect of an action, such as walking or jumping, as universally understood concepts. The key advantage of D-LORD is its ability to perform style transfer without needing paired motion data. Instead, it utilizes class and content labels during the latent optimization process. By disentangling the representation, the framework enables the transformation of one motion sequences style to another's style using Adaptive Instance Normalization. The proposed D-LORD framework is designed with a focus on generalization, allowing it to handle different class and content labels for various applications. Additionally, it can generate diverse motion sequences when specific class and content labels are provided. The framework's efficacy is demonstrated through experimentation on three datasets: the CMU XIA dataset for motion style transfer, the MHAD dataset, and the RRIS Ability dataset for motion retargeting. Notably, this paper presents the first generalized framework for motion style transfer and motion retargeting, showcasing its potential contributions in this area.
翻译:本文提出了一种名为D-LORD(双重潜在优化表示解缠)的新型框架,专为运动风格化(运动风格迁移与运动重定向)而设计。该框架的主要目标是通过数据驱动的潜在优化方法,从给定的运动序列中分离出类别信息与内容信息。其中,类别指代人物特定的风格,如特定情绪或个体身份;而内容则指动作中与风格无关的方面,如行走或跳跃这类普遍理解的概念。D-LORD的核心优势在于无需配对运动数据即可实现风格迁移,其通过在潜在优化过程中利用类别与内容标签达成此目的。通过解缠表示,该框架能够借助自适应实例归一化技术,将一个运动序列的风格转换为另一个风格。所提出的D-LORD框架注重泛化能力,可适配不同应用场景中的各类别与内容标签。此外,当给定特定类别与内容标签时,该框架能够生成多样化的运动序列。通过在三个数据集上的实验验证了框架的有效性:CMU XIA数据集用于运动风格迁移,MHAD数据集与RRIS Ability数据集用于运动重定向。值得注意的是,本文首次提出了适用于运动风格迁移与运动重定向的通用化框架,展现了其在该领域的潜在贡献。