Reconstructing the shape and appearance of real-world objects using measured 2D images has been a long-standing problem in computer vision. In this paper, we introduce a new analysis-by-synthesis technique capable of producing high-quality reconstructions through robust coarse-to-fine optimization and physics-based differentiable rendering. Unlike most previous methods that handle geometry and reflectance largely separately, our method unifies the optimization of both by leveraging image gradients with respect to both object reflectance and geometry. To obtain physically accurate gradient estimates, we develop a new GPU-based Monte Carlo differentiable renderer leveraging recent advances in differentiable rendering theory to offer unbiased gradients while enjoying better performance than existing tools like PyTorch3D and redner. To further improve robustness, we utilize several shape and material priors as well as a coarse-to-fine optimization strategy to reconstruct geometry. We demonstrate that our technique can produce reconstructions with higher quality than previous methods such as COLMAP and Kinect Fusion.

翻译：使用已测量的 2D 图像重建真实世界天体的形状和外观一直是计算机视觉中长期存在的一个问题。 在本文中,我们引入了一种新的分析与合成技术,能够通过强力粗到软优化和基于物理的不同成像来产生高质量的重建。 与以往处理几何和基本上分别反映的多数方法不同,我们的方法通过在物体反射和几何方面利用图像梯度来统一优化两者。 为了获得物理精确的梯度估计,我们开发了一个新的基于GPU的蒙特卡洛可变化工具,利用不同化理论的最新进展来提供不带偏见的梯度,同时比PyTorrch3D和Redner等现有工具更能提供更好的性能。 为了进一步提高稳健性,我们利用了多种形状和材料的前期以及粗到纤维的优化战略来重建几何。 我们证明我们的技术能够以比COLMAP 和Kinect Fusion等以往方法质量更高的方法产生重建。