Existing deep learning-based depth completion methods generally employ massive stacked layers to predict the dense depth map from sparse input data. Although such approaches greatly advance this task, their accompanied huge computational complexity hinders their practical applications. To accomplish depth completion more efficiently, we propose a novel lightweight deep network framework, the Long-short Range Recurrent Updating (LRRU) network. Without learning complex feature representations, LRRU first roughly fills the sparse input to obtain an initial dense depth map, and then iteratively updates it through learned spatially-variant kernels. Our iterative update process is content-adaptive and highly flexible, where the kernel weights are learned by jointly considering the guidance RGB images and the depth map to be updated, and large-to-small kernel scopes are dynamically adjusted to capture long-to-short range dependencies. Our initial depth map has coarse but complete scene depth information, which helps relieve the burden of directly regressing the dense depth from sparse ones, while our proposed method can effectively refine it to an accurate depth map with less learnable parameters and inference time. Experimental results demonstrate that our proposed LRRU variants achieve state-of-the-art performance across different parameter regimes. In particular, the LRRU-Base model outperforms competing approaches on the NYUv2 dataset, and ranks 1st on the KITTI depth completion benchmark at the time of submission. Project page: https://npucvr.github.io/LRRU/.

翻译：现有基于深度学习的深度补全方法通常采用大量堆叠层，从稀疏输入数据预测稠密深度图。虽然此类方法显著推进了该任务，但其伴随的巨大计算复杂度阻碍了实际应用。为更高效地实现深度补全，我们提出一种新颖的轻量级深度网络框架——长短程循环更新网络（LRRU）。无需学习复杂特征表示，LRRU首先粗填充稀疏输入以获得初始稠密深度图，然后通过学习的空间变化核对其进行迭代更新。我们的迭代更新过程具有内容自适应性和高度灵活性：核权重通过联合考虑引导RGB图像和待更新深度图学习得到，且大尺度到小尺度的核范围被动态调整以捕获长程到短程依赖关系。初始深度图虽粗糙但包含完整场景深度信息，这有助于减轻从稀疏数据直接回归稠密深度的负担；而所提方法可通过更少的可学习参数与推理时间将其有效精炼为精确深度图。实验结果表明，我们提出的LRRU变体在不同参数规模下均达到最先进性能。特别地，LRRU-Base模型在NYUv2数据集上超越竞争方法，并在提交时位列KITTI深度补全基准第一名。项目页面：https://npucvr.github.io/LRRU/。