Color-guided depth map super-resolution (CDSR) improve the spatial resolution of a low-quality depth map with the corresponding high-quality color map, benefiting various applications such as 3D reconstruction, virtual reality, and augmented reality. While conventional CDSR methods typically rely on convolutional neural networks or transformers, diffusion models (DMs) have demonstrated notable effectiveness in high-level vision tasks. In this work, we present a novel CDSR paradigm that utilizes a diffusion model within the latent space to generate guidance for depth map super-resolution. The proposed method comprises a guidance generation network (GGN), a depth map super-resolution network (DSRN), and a guidance recovery network (GRN). The GGN is specifically designed to generate the guidance while managing its compactness. Additionally, we integrate a simple but effective feature fusion module and a transformer-style feature extraction module into the DSRN, enabling it to leverage guided priors in the extraction, fusion, and reconstruction of multi-model images. Taking into account both accuracy and efficiency, our proposed method has shown superior performance in extensive experiments when compared to state-of-the-art methods. Our codes will be made available at https://github.com/shiyuan7/DSR-Diff.

翻译：彩色引导的深度图超分辨率（CDSR）通过利用对应的高质量彩色图提升低质量深度图的空间分辨率，惠及三维重建、虚拟现实和增强现实等多种应用。传统CDSR方法通常依赖卷积神经网络或Transformer，而扩散模型已在高层视觉任务中展现出显著效果。本文提出一种新颖的CDSR范式，在潜在空间内使用扩散模型生成引导信息，进而实现深度图超分辨率。所提方法由引导生成网络（GGN）、深度图超分辨率网络（DSRN）和引导恢复网络（GRN）三部分组成。其中GGN专为生成引导信息并控制其紧凑性而设计。此外，我们在DSRN中集成了简洁高效的特征融合模块与Transformer风格的特征提取模块，使其能够在多模态图像的提取、融合与重建过程中充分利用引导先验。在兼顾精度与效率的前提下，大量实验表明，所提方法相较于现有最优方法展现出卓越性能。相关代码将开源至https://github.com/shiyuan7/DSR-Diff。