Non-rigid structure-from-motion (NRSfM), a promising technique for addressing the mapping challenges in monocular visual deformable simultaneous localization and mapping (SLAM), has attracted growing attention. We introduce a novel method, called Con-NRSfM, for NRSfM under conformal deformations, encompassing isometric deformations as a subset. Our approach performs point-wise reconstruction using 2D selected image warps optimized through a graph-based framework. Unlike existing methods that rely on strict assumptions, such as locally planar surfaces or locally linear deformations, and fail to recover the conformal scale, our method eliminates these constraints and accurately computes the local conformal scale. Additionally, our framework decouples constraints on depth and conformal scale, which are inseparable in other approaches, enabling more precise depth estimation. To address the sensitivity of the formulated problem, we employ a parallel separable iterative optimization strategy. Furthermore, a self-supervised learning framework, utilizing an encoder-decoder network, is incorporated to generate dense 3D point clouds with texture. Simulation and experimental results using both synthetic and real datasets demonstrate that our method surpasses existing approaches in terms of reconstruction accuracy and robustness. The code for the proposed method will be made publicly available on the project website: https://sites.google.com/view/con-nrsfm.
翻译:非刚性运动恢复结构(NRSfM)作为一种有前景的技术,在解决单目视觉可变形同时定位与建图(SLAM)中的映射挑战方面日益受到关注。我们提出了一种名为Con-NRSfM的新方法,用于处理共形变形下的NRSfM问题,其涵盖等距变形作为一个子集。我们的方法通过基于图的框架优化二维选定图像形变,执行逐点三维重建。与现有方法依赖严格假设(如局部平面表面或局部线性变形)且无法恢复共形尺度不同,我们的方法消除了这些约束并精确计算了局部共形尺度。此外,我们的框架解耦了深度与共形尺度的约束条件——这些约束在其他方法中是不可分割的——从而实现了更精确的深度估计。为解决所构建问题的敏感性,我们采用了并行可分离迭代优化策略。进一步地,我们结合了一个利用编码器-解码器网络的自监督学习框架,以生成带纹理的稠密三维点云。使用合成和真实数据集的仿真与实验结果表明,我们的方法在重建精度和鲁棒性方面均优于现有方法。所提方法的代码将在项目网站上公开:https://sites.google.com/view/con-nrsfm。