Depth estimation attracts widespread attention in the computer vision community. However, it is still quite difficult to recover an accurate depth map using only one RGB image. We observe a phenomenon that existing methods tend to fail in different cases, caused by differences in network architecture, loss function and so on. In this work, we investigate into the phenomenon and propose to integrate the strengths of multiple weak depth predictor to build a comprehensive and accurate depth predictor, which is critical for many real-world applications, e.g., 3D reconstruction. Specifically, we construct multiple base (weak) depth predictors by utilizing different Transformer-based and convolutional neural network (CNN)-based architectures. Transformer establishes long-range correlation while CNN preserves local information ignored by Transformer due to the spatial inductive bias. Therefore, the coupling of Transformer and CNN contributes to the generation of complementary depth estimates, which are essential to achieve a comprehensive depth predictor. Then, we design mixers to learn from multiple weak predictions and adaptively fuse them into a strong depth estimate. The resultant model, which we refer to as Transformer-assisted depth ensembles (TEDepth). On the standard NYU-Depth-v2 and KITTI datasets, we thoroughly explore how the neural ensembles affect the depth estimation and demonstrate that our TEDepth achieves better results than previous state-of-the-art approaches. To validate the generalizability across cameras, we directly apply the models trained on NYU-Depth-v2 to the SUN RGB-D dataset without any fine-tuning, and the superior results emphasize its strong generalizability.

翻译：深度估计在计算机视觉领域引起了广泛关注。然而，仅凭一张RGB图像恢复出精确的深度图仍然相当困难。我们观察到一种现象：现有方法因网络架构、损失函数等差异，往往在不同情况下失效。在本工作中，我们研究了这一现象，并提议整合多个弱深度预测器的优势，构建一个全面且精确的深度预测器——这对于许多现实应用（如三维重建）至关重要。具体而言，我们通过利用不同的基于Transformer和卷积神经网络（CNN）的架构构建多个基础（弱）深度预测器。Transformer能够建立长程相关性，而CNN则保留了因空间归纳偏置而被Transformer忽略的局部信息。因此，Transformer与CNN的耦合有助于生成互补的深度估计，这对实现全面的深度预测器至关重要。随后，我们设计了混合器，用于从多个弱预测中学习，并将其自适应融合为强深度估计。由此产生的模型被称为变压器辅助深度集成（TEDepth）。在标准NYU-Depth-v2和KITTI数据集上，我们深入探讨了神经集成如何影响深度估计，并证明了我们的TEDepth优于以往最先进的方法。为了验证模型在不同相机上的泛化能力，我们将在NYU-Depth-v2上训练的模型直接应用于SUN RGB-D数据集，无需任何微调，其优越的结果凸显了其强大的泛化能力。