Inverse rendering of outdoor scenes from unconstrained image collections is a challenging task, particularly illumination/albedo ambiguities and occlusion of the illumination environment (shadowing) caused by geometry. However, there are many cues in an image that can aid in the disentanglement of geometry, albedo and shadows. We exploit the fact that any sky pixel provides a direct measurement of distant lighting in the corresponding direction and, via a neural illumination prior, a statistical cue as to the remaining illumination environment. We also introduce a novel `outside-in' method for computing differentiable sky visibility based on a neural directional distance function. This is efficient and can be trained in parallel with the neural scene representation, allowing gradients from appearance loss to flow from shadows to influence estimation of illumination and geometry. Our method estimates high-quality albedo, geometry, illumination and sky visibility, achieving state-of-the-art results on the NeRF-OSR relighting benchmark. Our code and models can be found https://github.com/JADGardner/neusky

翻译：从非约束图像集合中对室外场景进行逆渲染是一项极具挑战的任务，尤其是光照/反照率模糊性以及几何结构对光照环境（阴影）造成的遮挡问题。然而，图像中存在诸多可辅助分离几何、反照率与阴影的线索。我们利用以下事实：任意天空像素能直接提供对应方向的远距离光照测量值，并通过神经光照先验获取剩余光照环境的统计线索。同时，我们提出了一种基于神经定向距离函数的"由外向内"可微分天空可见性计算方法。该方法高效且能与神经场景表示并行训练，使得外观损失梯度可从阴影传播至光照与几何估计。本方法可估计高质量的反照率、几何、光照及天空可见性，在NeRF-OSR重照明基准上达到了当前最优性能。我们的代码与模型见：https://github.com/JADGardner/neusky