Self-supervised monocular depth estimation methods have been increasingly given much attention due to the benefit of not requiring large, labelled datasets. Such self-supervised methods require high-quality salient features and consequently suffer from severe performance drop for indoor scenes, where low-textured regions dominant in the scenes are almost indiscriminative. To address the issue, we propose a self-supervised indoor monocular depth estimation framework called $\mathrm{F^2Depth}$. A self-supervised optical flow estimation network is introduced to supervise depth learning. To improve optical flow estimation performance in low-textured areas, only some patches of points with more discriminative features are adopted for finetuning based on our well-designed patch-based photometric loss. The finetuned optical flow estimation network generates high-accuracy optical flow as a supervisory signal for depth estimation. Correspondingly, an optical flow consistency loss is designed. Multi-scale feature maps produced by finetuned optical flow estimation network perform warping to compute feature map synthesis loss as another supervisory signal for depth learning. Experimental results on the NYU Depth V2 dataset demonstrate the effectiveness of the framework and our proposed losses. To evaluate the generalization ability of our $\mathrm{F^2Depth}$, we collect a Campus Indoor depth dataset composed of approximately 1500 points selected from 99 images in 18 scenes. Zero-shot generalization experiments on 7-Scenes dataset and Campus Indoor achieve $\delta_1$ accuracy of 75.8% and 76.0% respectively. The accuracy results show that our model can generalize well to monocular images captured in unknown indoor scenes.
翻译:自监督单目深度估计方法因无需大量标注数据集而日益受到关注。此类自监督方法需要高质量显著性特征,因此在以低纹理区域为主导且特征近乎无差异的室内场景中,性能会大幅下降。为解决此问题,我们提出了一种名为$\mathrm{F^2Depth}$的自监督室内单目深度估计框架。引入自监督光流估计网络来监督深度学习。为提升低纹理区域的光流估计性能,基于精心设计的基于块的光度损失,仅选取具有更具区分性特征的部分点块进行微调。微调后的光流估计网络生成高精度光流,作为深度估计的监督信号,并据此设计光流一致性损失。微调光流估计网络生成的多尺度特征图通过扭曲操作计算特征图合成损失,作为深度学习的另一监督信号。在NYU Depth V2数据集上的实验结果表明了该框架及所提损失函数的有效性。为评估$\mathrm{F^2Depth}$的泛化能力,我们构建了校园室内深度数据集,该数据集包含来自18个场景99幅图像的约1500个采样点。在7-Scenes数据集和校园室内数据集上的零样本泛化实验中,$\delta_1$精度分别达到75.8%和76.0%,表明模型可良好泛化至未知室内场景中采集的单目图像。