To solve the problem of pose distortion in the forward propagation of pose features in existing methods, this pa-per proposes a Dual-Side Feature Fusion Network for pose transfer (DSFFNet). Firstly, a fixed-length pose code is extracted from the source mesh by a pose encoder and combined with the target vertices to form a mixed feature; Then, a Feature Fusion Adaptive Instance Normalization module (FFAdaIN) is designed, which can process both pose and identity features simultaneously, so that the pose features can be compensated in layer-by-layer for-ward propagation, thus solving the pose distortion problem; Finally, using the mesh decoder composed of this module, the pose are gradually transferred to the target mesh. Experimental results on SMPL, SMAL, FAUST and MultiGarment datasets show that DSFFNet successfully solves the pose distortion problem while maintaining a smaller network structure with stronger pose transfer capability and faster convergence speed, and can adapt to meshes with different numbers of vertices. Code is available at https://github.com/YikiDragon/DSFFNet
翻译:为解决现有方法中姿态特征前向传播时出现的姿态畸变问题,本文提出了一种用于姿态迁移的双侧特征融合网络(DSFFNet)。首先,通过姿态编码器从源网格中提取固定长度的姿态码,并与目标顶点结合形成混合特征;其次,设计了特征融合自适应实例归一化模块(FFAdaIN),该模块能够同时处理姿态和身份特征,使姿态特征在逐层前向传播中得到补偿,从而解决姿态畸变问题;最后,利用该模块组成的网格解码器,将姿态逐步迁移至目标网格。在SMPL、SMAL、FAUST和MultiGarment数据集上的实验结果表明,DSFFNet在成功解决姿态畸变问题的同时,保持了更小的网络结构,具备更强的姿态迁移能力和更快的收敛速度,并且能够适配不同顶点数量的网格。代码已在https://github.com/YikiDragon/DSFFNet 开源。