We present Deep Shape-from-Template (DeepSfT), a novel Deep Neural Network (DNN) method for solving real-time automatic registration and 3D reconstruction of a deformable object viewed in a single monocular image.DeepSfT advances the state-of-the-art in various aspects. Compared to existing DNN SfT methods, it is the first fully convolutional real-time approach that handles an arbitrary object geometry, topology and surface representation. It also does not require ground truth registration with real data and scales well to very complex object models with large numbers of elements. Compared to previous non-DNN SfT methods, it does not involve numerical optimization at run-time, and is a dense, wide-baseline solution that does not demand, and does not suffer from, feature-based matching. It is able to process a single image with significant deformation and viewpoint changes, and handles well the core challenges of occlusions, weak texture and blur. DeepSfT is based on residual encoder-decoder structures and refining blocks. It is trained end-to-end with a novel combination of supervised learning from simulated renderings of the object model and semi-supervised automatic fine-tuning using real data captured with a standard RGB-D camera. The cameras used for fine-tuning and run-time can be different, making DeepSfT practical for real-world use. We show that DeepSfT significantly outperforms state-of-the-art wide-baseline approaches for non-trivial templates, with quantitative and qualitative evaluation.

翻译：我们展示了从Template (DeepSfT) 的Deep 形状(DeepSfT), 这是用于解决实时自动登记和3D重建一个在单个单层图像中查看的变形对象的新颖的深深神经网络(DNNN) 的方法。 与现有的 DNNN SfT 方法相比, 这是第一个完全渐进实时方法, 处理任意对象的几何、 地形学和表面代表。 它也不需要用真实数据和比例进行地面真实的真象登记, 更接近具有大量深元素的非常复杂的物体模型。 与以前的非 DNNNSfT 方法相比, 它不涉及运行时的数值优化, 并且是一个不要求、 不因地基匹配而受到影响的宽广基解决方案。 它能够处理一个具有显著变形和视觉变化的单一图像, 并且能够很好地处理隐形、 软质的纹理和模糊的核心挑战。 深SfT 是基于对不精密的解解码和精细的图像结构的剩余组合, 和精细的精细的智能智能智能智能智能的S- 将S- 的Sloveal- d- dal- dal- dal- dal- dal- disal- d- disal- disal- disal- d- disal- disal- di- d- disal- di- disal- di- di- di- di- di- di- sal- di- sal- di- di- di- di- sal- sal- sal- sal- to- sal- sal-d- sal- sal-d- to- to- sal- sal- to-dal-d-d-d- sal- sal- sal- sal-dal- to- to- to- to-dal-dal-dal- sal- sal-d-d-d- to- sal- sal- to- sal- sal-d- to- to- sal- sal- sal- sal- sal- sal- sal- to-d- to- sal- to-