Recent RGB-guided depth super-resolution methods have achieved impressive performance under the assumption of fixed and known degradation (e.g., bicubic downsampling). However, in real-world scenarios, captured depth data often suffer from unconventional and unknown degradation due to sensor limitations and complex imaging environments (e.g., low reflective surfaces, varying illumination). Consequently, the performance of these methods significantly declines when real-world degradation deviate from their assumptions. In this paper, we propose the Degradation Oriented and Regularized Network (DORNet), a novel framework designed to adaptively address unknown degradation in real-world scenes through implicit degradation representations. Our approach begins with the development of a self-supervised degradation learning strategy, which models the degradation representations of low-resolution depth data using routing selection-based degradation regularization. To facilitate effective RGB-D fusion, we further introduce a degradation-oriented feature transformation module that selectively propagates RGB content into the depth data based on the learned degradation priors. Extensive experimental results on both real and synthetic datasets demonstrate the superiority of our DORNet in handling unknown degradation, outperforming existing methods. The code is available at https://github.com/yanzq95/DORNet.
翻译:近年来,基于RGB引导的深度超分辨率方法在固定且已知的退化(如双三次下采样)假设下取得了显著性能。然而,在实际场景中,由于传感器限制和复杂成像环境(如低反射表面、变化光照),采集的深度数据常遭受非常规且未知的退化。因此,当实际退化偏离其假设时,这些方法的性能会显著下降。本文提出面向退化与正则化的网络(DORNet),这是一种通过隐式退化表示自适应处理真实场景中未知退化的新型框架。我们的方法首先开发了一种自监督退化学习策略,该策略利用基于路由选择的退化正则化对低分辨率深度数据的退化表示进行建模。为促进有效的RGB-D融合,我们进一步引入了面向退化的特征转换模块,该模块根据学习到的退化先验,有选择地将RGB内容传播至深度数据中。在真实与合成数据集上的大量实验结果证明了我们的DORNet在处理未知退化方面的优越性,其性能超越了现有方法。代码发布于https://github.com/yanzq95/DORNet。