We present As-Plausible-as-Possible (APAP) mesh deformation technique that leverages 2D diffusion priors to preserve the plausibility of a mesh under user-controlled deformation. Our framework uses per-face Jacobians to represent mesh deformations, where mesh vertex coordinates are computed via a differentiable Poisson Solve. The deformed mesh is rendered, and the resulting 2D image is used in the Score Distillation Sampling (SDS) process, which enables extracting meaningful plausibility priors from a pretrained 2D diffusion model. To better preserve the identity of the edited mesh, we fine-tune our 2D diffusion model with LoRA. Gradients extracted by SDS and a user-prescribed handle displacement are then backpropagated to the per-face Jacobians, and we use iterative gradient descent to compute the final deformation that balances between the user edit and the output plausibility. We evaluate our method with 2D and 3D meshes and demonstrate qualitative and quantitative improvements when using plausibility priors over geometry-preservation or distortion-minimization priors used by previous techniques.
翻译:我们提出了一种尽可能合理(APAP)网格变形技术,该技术利用2D扩散先验来保持用户控制变形下网格的合理性。我们的框架使用逐面雅可比矩阵表示网格变形,通过可微泊松求解计算网格顶点坐标。对变形网格进行渲染,并将生成的2D图像用于分数蒸馏采样(SDS)过程,从而从预训练的2D扩散模型中提取有意义的合理性先验。为了更好地保持编辑网格的同一性,我们使用LoRA对2D扩散模型进行微调。由SDS提取的梯度和用户指定的手柄位移随后反向传播至逐面雅可比矩阵,我们通过迭代梯度下降法计算在用户编辑与输出合理性之间取得平衡的最终变形。我们使用2D和3D网格评估了该方法,并证明了相较于以往技术所使用的几何保持或失真最小化先验,采用合理性先验在定性和定量上均具有改进效果。