Recent studies construct deblurred neural radiance fields (DeRF) using dozens of blurry images, which are not practical scenarios if only a limited number of blurry images are available. This paper focuses on constructing DeRF from sparse-view for more pragmatic real-world scenarios. As observed in our experiments, establishing DeRF from sparse views proves to be a more challenging problem due to the inherent complexity arising from the simultaneous optimization of blur kernels and NeRF from sparse view. Sparse-DeRF successfully regularizes the complicated joint optimization, presenting alleviated overfitting artifacts and enhanced quality on radiance fields. The regularization consists of three key components: Surface smoothness, helps the model accurately predict the scene structure utilizing unseen and additional hidden rays derived from the blur kernel based on statistical tendencies of real-world; Modulated gradient scaling, helps the model adjust the amount of the backpropagated gradient according to the arrangements of scene objects; Perceptual distillation improves the perceptual quality by overcoming the ill-posed multi-view inconsistency of image deblurring and distilling the pre-filtered information, compensating for the lack of clean information in blurry images. We demonstrate the effectiveness of the Sparse-DeRF with extensive quantitative and qualitative experimental results by training DeRF from 2-view, 4-view, and 6-view blurry images.
翻译:近期研究利用数十张模糊图像构建去模糊神经辐射场(DeRF),但在仅能获取有限模糊图像的实际场景中缺乏实用性。本文聚焦于从稀疏视图构建DeRF,以应对更贴近现实的场景需求。实验发现,由于需要同时优化模糊核与稀疏视图下的神经辐射场(NeRF),从稀疏视图建立DeRF成为更具挑战性的问题。Sparse-DeRF通过有效的正则化机制成功约束了这一复杂的联合优化过程,显著减轻了过拟合伪影并提升了辐射场质量。该正则化框架包含三个核心组件:表面平滑性约束——基于现实场景的统计规律,利用模糊核衍生的未见光线与附加隐藏光线,帮助模型精确预测场景结构;调制梯度缩放——根据场景物体空间排布动态调整反向传播梯度量;感知蒸馏——通过克服图像去模糊中病态的多视角不一致性,并蒸馏预滤波信息来提升感知质量,从而补偿模糊图像中清晰信息的缺失。我们通过使用2视图、4视图和6视图模糊图像训练DeRF,以大量定量与定性实验结果验证了Sparse-DeRF的有效性。