Given sparse depths and the corresponding RGB images, depth completion aims at spatially propagating the sparse measurements throughout the whole image to get a dense depth prediction. Despite the tremendous progress of deep-learning-based depth completion methods, the locality of the convolutional layer or graph model makes it hard for the network to model the long-range relationship between pixels. While recent fully Transformer-based architecture has reported encouraging results with the global receptive field, the performance and efficiency gaps to the well-developed CNN models still exist because of its deteriorative local feature details. This paper proposes a Joint Convolutional Attention and Transformer block (JCAT), which deeply couples the convolutional attention layer and Vision Transformer into one block, as the basic unit to construct our depth completion model in a pyramidal structure. This hybrid architecture naturally benefits both the local connectivity of convolutions and the global context of the Transformer in one single model. As a result, our CompletionFormer outperforms state-of-the-art CNNs-based methods on the outdoor KITTI Depth Completion benchmark and indoor NYUv2 dataset, achieving significantly higher efficiency (nearly 1/3 FLOPs) compared to pure Transformer-based methods. Code is available at \url{https://github.com/youmi-zym/CompletionFormer}.

翻译：给定稀疏深度图和对应的RGB图像，深度补全旨在将稀疏测量值在空间上沿整个图像传播，以获取稠密深度预测。尽管基于深度学习的深度补全方法取得了显著进展，但卷积层或图模型的局部性使得网络难以建模像素间的长程关系。近期全Transformer架构虽凭借全局感受野取得了令人鼓舞的结果，但由于局部特征细节退化，其在性能和效率上仍与成熟的CNN模型存在差距。本文提出联合卷积注意力与Transformer模块（JCAT），将卷积注意力层与视觉Transformer深度耦合为一个基本单元，并以金字塔结构构建深度补全模型。这种混合架构自然融合了卷积的局部连通性与Transformer的全局上下文能力。实验表明，本文提出的CompletionFormer在室外KITTI深度补全基准和室内NYUv2数据集上均优于基于CNN的现有方法，同时相比纯Transformer方法实现了显著更高的效率（FLOPs降低近1/3）。代码开源于\url{https://github.com/youmi-zym/CompletionFormer}。