Depth super-resolution (DSR) aims to restore high-resolution (HR) depth from low-resolution (LR) one, where RGB image is often used to promote this task. Recent image guided DSR approaches mainly focus on spatial domain to rebuild depth structure. However, since the structure of LR depth is usually blurry, only considering spatial domain is not very sufficient to acquire satisfactory results. In this paper, we propose structure guided network (SGNet), a method that pays more attention to gradient and frequency domains, both of which have the inherent ability to capture high-frequency structure. Specifically, we first introduce the gradient calibration module (GCM), which employs the accurate gradient prior of RGB to sharpen the LR depth structure. Then we present the Frequency Awareness Module (FAM) that recursively conducts multiple spectrum differencing blocks (SDB), each of which propagates the precise high-frequency components of RGB into the LR depth. Extensive experimental results on both real and synthetic datasets demonstrate the superiority of our SGNet, reaching the state-of-the-art. Codes and pre-trained models are available at https://github.com/yanzq95/SGNet.

翻译：深度超分辨率（DSR）旨在从低分辨率（LR）深度图中恢复高分辨率（HR）深度图，其中RGB图像常被用于辅助该任务。现有基于图像引导的DSR方法主要关注空间域以重建深度结构。然而，由于LR深度图的结构通常较为模糊，仅考虑空间域不足以获得令人满意的结果。本文提出结构引导网络（SGNet），该方法更关注梯度域和频率域，两者均天然具备捕捉高频结构的能力。具体而言，我们首先引入梯度校准模块（GCM），利用RGB图像的精确梯度先验锐化LR深度图结构；随后提出频率感知模块（FAM），通过递归执行多个频谱差分块（SDB），将RGB图像的精确高频分量传播至LR深度图。在真实与合成数据集上的大量实验结果表明，我们的SGNet达到了当前最优性能。代码与预训练模型已开源至 https://github.com/yanzq95/SGNet。