Video-guided 3D animation holds immense potential for content creation, offering intuitive and precise control over dynamic assets. However, practical deployment faces a critical yet frequently overlooked hurdle: the pose misalignment dilemma. In real-world scenarios, the initial pose of a user-provided static mesh rarely aligns with the starting frame of a reference video. Naively forcing a mesh to follow a mismatched trajectory inevitably leads to severe geometric distortion or animation failure. To address this, we present Rectified Dynamic Mesh (R-DMesh), a unified framework designed to generate high-fidelity 4D meshes that are ``rectified'' to align with video context. Unlike standard motion transfer approaches, our method introduces a novel VAE that explicitly disentangles the input into a conditional base mesh, relative motion trajectories, and a crucial rectification jump offset. This offset is learned to automatically transform the arbitrary pose of the input mesh to match the video's initial state before animation begins. We process these components via a Triflow Attention mechanism, which leverages vertex-wise geometric features to modulate the three orthogonal flows, ensuring physical consistency and local rigidity during the rectification and animation process. For generation, we employ a Rectified Flow-based Diffusion Transformer conditioned on pre-trained video latents, effectively transferring rich spatio-temporal priors to the 3D domain. To support this task, we construct Video-RDMesh, a large-scale dataset of over 500k dynamic mesh sequences specifically curated to simulate pose misalignment. Extensive experiments demonstrate that R-DMesh not only solves the alignment problem but also enables robust downstream applications, including pose retargeting and holistic 4D generation.
翻译:视频驱动三维动画在内容创作中具有巨大潜力,可为动态资产提供直观且精确的控制。然而,实际部署面临一个关键且常被忽视的障碍:姿态错位困境。在真实场景中,用户提供的静态网格初始姿态与参考视频起始帧难以对齐。若强行令网格跟随不匹配的运动轨迹,将不可避免地导致严重几何畸变或动画生成失败。为此,我们提出修正动态网格(R-DMesh)——一个旨在生成与视频上下文对齐的"修正"高保真四维网格的统一框架。与标准运动迁移方法不同,本方法引入新型变分自编码器,将输入显式解耦为条件基础网格、相对运动轨迹以及关键的修正跳跃偏移量。该偏移量经学习后可自动将输入网格的任意姿态变换至视频初始状态,再启动动画生成。我们通过三流注意力机制处理这些分量,该机制利用顶点级几何特征调控三个正交流,确保修正与动画过程中的物理一致性与局部刚性。在生成阶段,我们采用基于修正流的扩散Transformer,以预训练视频潜变量为条件,将丰富的时空先验有效迁移至三维域。为支撑该任务,我们构建了Video-RDMesh——包含超50万动态网格序列的大规模数据集,专门设计以模拟姿态错位场景。大量实验证明,R-DMesh不仅解决了对齐问题,更实现了鲁棒的下游应用,包括姿态重定向与整体四维生成。