We propose PRM, a novel photometric stereo based large reconstruction model to reconstruct high-quality meshes with fine-grained local details. Unlike previous large reconstruction models that prepare images under fixed and simple lighting as both input and supervision, PRM renders photometric stereo images by varying materials and lighting for the purposes, which not only improves the precise local details by providing rich photometric cues but also increases the model robustness to variations in the appearance of input images. To offer enhanced flexibility of images rendering, we incorporate a real-time physically-based rendering (PBR) method and mesh rasterization for online images rendering. Moreover, in employing an explicit mesh as our 3D representation, PRM ensures the application of differentiable PBR, which supports the utilization of multiple photometric supervisions and better models the specular color for high-quality geometry optimization. Our PRM leverages photometric stereo images to achieve high-quality reconstructions with fine-grained local details, even amidst sophisticated image appearances. Extensive experiments demonstrate that PRM significantly outperforms other models.

翻译：我们提出PRM，一种基于光度立体的新型大型重建模型，用于重建具有精细局部细节的高质量网格。与以往采用固定简单光照下图像作为输入和监督的大型重建模型不同，PRM通过变化材质和光照渲染光度立体图像，这不仅通过提供丰富的光度线索提升了局部细节的精确性，还增强了模型对输入图像外观变化的鲁棒性。为实现更灵活的图像渲染，我们结合了实时物理渲染方法与网格光栅化技术进行在线图像渲染。此外，通过采用显式网格作为三维表示，PRM确保了可微分物理渲染的应用，从而支持利用多重光度监督并更好地建模高光颜色以实现高质量的几何优化。我们的PRM利用光度立体图像，即使在复杂图像外观条件下，仍能实现具有精细局部细节的高质量重建。大量实验表明，PRM显著优于其他模型。