Photometric stereo leverages variations in illumination conditions to reconstruct per-pixel surface normals. The concept of display photometric stereo, which employs a conventional monitor as an illumination source, has the potential to overcome limitations often encountered in bulky and difficult-to-use conventional setups. In this paper, we introduce Differentiable Display Photometric Stereo (DDPS), a method designed to achieve high-fidelity normal reconstruction using an off-the-shelf monitor and camera. DDPS addresses a critical yet often neglected challenge in photometric stereo: the optimization of display patterns for enhanced normal reconstruction. We present a differentiable framework that couples basis-illumination image formation with a photometric-stereo reconstruction method. This facilitates the learning of display patterns that leads to high-quality normal reconstruction through automatic differentiation. Addressing the synthetic-real domain gap inherent in end-to-end optimization, we propose the use of a real-world photometric-stereo training dataset composed of 3D-printed objects. Moreover, to reduce the ill-posed nature of photometric stereo, we exploit the linearly polarized light emitted from the monitor to optically separate diffuse and specular reflections in the captured images. We demonstrate that DDPS allows for learning display patterns optimized for a target configuration and is robust to initialization. We assess DDPS on 3D-printed objects with ground-truth normals and diverse real-world objects, validating that DDPS enables effective photometric-stereo reconstruction.

翻译：光度立体技术利用光照条件的变化来重建每个像素的表面法线。显示光度立体这一概念采用传统显示器作为照明源，具有克服传统笨重且难以使用的设备常见局限的潜力。本文提出可微分显示光度立体（DDPS），这是一种旨在使用现成显示器和相机实现高保真法线重建的方法。DDPS解决了光度立体中一个关键但常被忽视的挑战：优化显示图案以增强法线重建。我们提出一个可微分框架，将基照明图像形成过程与光度立体重建方法相结合。这通过自动微分促进了能够实现高质量法线重建的显示图案学习。针对端到端优化中固有的合成-真实领域差距，我们提出使用由3D打印物体组成的真实世界光度立体训练数据集。此外，为了减少光度立体的病态特性，我们利用显示器发出的线偏振光，在捕获图像中光学分离漫反射和镜面反射成分。我们证明DDPS能够学习针对特定配置优化的显示图案，且对初始化具有鲁棒性。我们在具有真实法线的3D打印物体和多种真实世界物体上评估DDPS，验证了DDPS能够实现有效的光度立体重建。