Structured light has proven instrumental in 3D imaging, LiDAR, and holographic light projection. Metasurfaces, comprised of sub-wavelength-sized nanostructures, facilitate 180$^\circ$ field-of-view (FoV) structured light, circumventing the restricted FoV inherent in traditional optics like diffractive optical elements. However, extant metasurface-facilitated structured light exhibits sub-optimal performance in downstream tasks, due to heuristic pattern designs such as periodic dots that do not consider the objectives of the end application. In this paper, we present neural 360$^\circ$ structured light, driven by learned metasurfaces. We propose a differentiable framework, that encompasses a computationally-efficient 180$^\circ$ wave propagation model and a task-specific reconstructor, and exploits both transmission and reflection channels of the metasurface. Leveraging a first-order optimizer within our differentiable framework, we optimize the metasurface design, thereby realizing neural 360$^\circ$ structured light. We have utilized neural 360$^\circ$ structured light for holographic light projection and 3D imaging. Specifically, we demonstrate the first 360$^\circ$ light projection of complex patterns, enabled by our propagation model that can be computationally evaluated 50,000$\times$ faster than the Rayleigh-Sommerfeld propagation. For 3D imaging, we improve depth-estimation accuracy by 5.09$\times$ in RMSE compared to the heuristically-designed structured light. Neural 360$^\circ$ structured light promises robust 360$^\circ$ imaging and display for robotics, extended-reality systems, and human-computer interactions.

翻译：结构光在三维成像、激光雷达和全息光投影中已证明其重要价值。由亚波长纳米结构组成的超表面可实现180°视场角的结构光，突破了传统光学元件（如衍射光学元件）固有的视场限制。然而，现有基于超表面的结构光由于采用启发式图案设计（如周期性点阵），未考虑最终应用目标，在下游任务中表现欠佳。本文提出基于学习型超表面的神经360°结构光方案。我们构建了一个可微分框架，包含计算高效的180°波传播模型和任务特定重构器，并同时利用超表面的透射通道与反射通道。通过在该可微分框架中应用一阶优化器对超表面设计进行优化，最终实现了神经360°结构光。我们将所提方法应用于全息光投影与三维成像：在全息投影方面，首次实现了复杂图案的360°光投影，所提出的传播模型计算速度比瑞利-索末菲传播模型快50,000倍；在三维成像方面，与启发式设计的结构光相比，深度估计的均方根误差降低了5.09倍。神经360°结构光为机器人、扩展现实系统及人机交互提供了鲁棒的360°成像与显示方案。