In this paper, we propose a novel method for joint recovery of camera pose, object geometry and spatially-varying Bidirectional Reflectance Distribution Function (svBRDF) of 3D scenes that exceed object-scale and hence cannot be captured with stationary light stages. The input are high-resolution RGB-D images captured by a mobile, hand-held capture system with point lights for active illumination. Compared to previous works that jointly estimate geometry and materials from a hand-held scanner, we formulate this problem using a single objective function that can be minimized using off-the-shelf gradient-based solvers. To facilitate scalability to large numbers of observation views and optimization variables, we introduce a distributed optimization algorithm that reconstructs 2.5D keyframe-based representations of the scene. A novel multi-view consistency regularizer effectively synchronizes neighboring keyframes such that the local optimization results allow for seamless integration into a globally consistent 3D model. We provide a study on the importance of each component in our formulation and show that our method compares favorably to baselines. We further demonstrate that our method accurately reconstructs various objects and materials and allows for expansion to spatially larger scenes. We believe that this work represents a significant step towards making geometry and material estimation from hand-held scanners scalable.

翻译：本文提出了一种新方法，用于联合恢复超出物体尺度（因而无法通过静态光照舞台捕获）的三维场景的相机位姿、物体几何以及空间变化的双向反射分布函数（svBRDF）。输入数据为由手持移动捕获系统（配备点光源进行主动照明）采集的高分辨率RGB-D图像。与先前从手持扫描仪联合估计几何与材质的工作不同，本文将问题建模为可借助现成梯度求解器进行最小化的单一目标函数。为支持对大量观测视图和优化变量的可扩展性，我们引入了一种分布式优化算法，用于重建场景的2.5D关键帧表示。一种新颖的多视图一致性正则化器有效同步了相邻关键帧，使得局部优化结果能够无缝集成到全局一致的三维模型中。我们评估了公式中每个组成部分的重要性，并表明所提方法优于基线方法。进一步实验证明，该方法能准确重建多种物体与材质，并可扩展至空间维度更大的场景。我们相信，这项工作是推动从手持扫描仪可扩展地估计几何与材质的重要进展。