Existing time-resolved non-line-of-sight (NLOS) imaging methods reconstruct hidden scenes by inverting the optical paths of indirect illumination measured at visible relay surfaces. These methods are prone to reconstruction artifacts due to inversion ambiguities and capture noise, which are typically mitigated through the manual selection of filtering functions and parameters. We introduce a fully-differentiable end-to-end NLOS inverse rendering pipeline that self-calibrates the imaging parameters during the reconstruction of hidden scenes, using as input only the measured illumination while working both in the time and frequency domains. Our pipeline extracts a geometric representation of the hidden scene from NLOS volumetric intensities and estimates the time-resolved illumination at the relay wall produced by such geometric information using differentiable transient rendering. We then use gradient descent to optimize imaging parameters by minimizing the error between our simulated time-resolved illumination and the measured illumination. Our end-to-end differentiable pipeline couples diffraction-based volumetric NLOS reconstruction with path-space light transport and a simple ray marching technique to extract detailed, dense sets of surface points and normals of hidden scenes. We demonstrate the robustness of our method to consistently reconstruct geometry and albedo, even under significant noise levels.

翻译：现有时间分辨非视距成像方法通过反演可见中继表面测得的间接光照光学路径来重建隐藏场景。由于反演模糊性和捕获噪声，这些方法易产生重建伪影，通常需通过手动选择滤波函数和参数来缓解。我们提出一种全微分端到端非视距逆渲染管线，该管线在隐藏场景重建过程中能自校准成像参数，仅以测量光照为输入，并在时域和频域同时工作。该管线从非视具体积强度中提取隐藏场景的几何表示，并通过可微分瞬态渲染估计该几何信息在中继壁面产生的时间分辨光照。随后利用梯度下降法，通过最小化模拟时间分辨光照与测量光照之间的误差来优化成像参数。我们的端到端可微分管线将基于衍射的体积非视距重建与路径空间光传输及简单光线步进技术相结合，可提取隐藏场景的密集详细表面点集与法向量。实验表明，即使在显著噪声水平下，该方法也能稳定重建几何与反照率。