Photometric stereo leverages variations in illumination conditions to reconstruct surface normals. Display photometric stereo, which employs a conventional monitor as an illumination source, has the potential to overcome limitations often encountered in bulky and difficult-to-use conventional setups. In this paper, we present differentiable display photometric stereo (DDPS), addressing an often overlooked challenge in display photometric stereo: the design of display patterns. Departing from using heuristic display patterns, DDPS learns the display patterns that yield accurate normal reconstruction for a target system in an end-to-end manner. To this end, we propose a differentiable framework that couples basis-illumination image formation with analytic photometric-stereo reconstruction. The differentiable framework facilitates the effective learning of display patterns via auto-differentiation. Also, for training supervision, we propose to use 3D printing for creating a real-world training dataset, enabling accurate reconstruction on the target real-world setup. Finally, we exploit that conventional LCD monitors emit polarized light, which allows for the optical separation of diffuse and specular reflections when combined with a polarization camera, leading to accurate normal reconstruction. Extensive evaluation of DDPS shows improved normal-reconstruction accuracy compared to heuristic patterns and demonstrates compelling properties such as robustness to pattern initialization, calibration errors, and simplifications in image formation and reconstruction.

翻译：光度立体法利用光照条件的变化来重建表面法线。显示光度立体法采用常规显示器作为光源，有潜力克服传统装置通常存在的体积庞大且难以操作的局限。本文提出可微显示光度立体（DDPS），解决显示光度立体中一个常被忽视的挑战：显示图案的设计。不同于使用启发式显示图案，DDPS以端到端方式学习能为目标系统产生准确法线重建的显示图案。为此，我们提出一个可微框架，将基照明图像形成与分析光度立体重建相结合。该可微框架通过自动微分促进了显示图案的有效学习。此外，为训练监督，我们提议使用3D打印创建真实世界训练数据集，从而在目标真实设置上实现准确重建。最后，我们利用常规LCD显示器发射偏振光这一特性，结合偏振相机实现漫反射与镜面反射的光学分离，进而实现准确法线重建。对DDPS的广泛评估表明，相比启发式图案，其法线重建精度得到提升，并展现出对图案初始化、标定误差以及图像形成与重建简化的鲁棒性等引人注目的特性。