Neural-network-based single image depth prediction (SIDP) is a challenging task where the goal is to predict the scene's per-pixel depth at test time. Since the problem, by definition, is ill-posed, the fundamental goal is to come up with an approach that can reliably model the scene depth from a set of training examples. In the pursuit of perfect depth estimation, most existing state-of-the-art learning techniques predict a single scalar depth value per-pixel. Yet, it is well-known that the trained model has accuracy limits and can predict imprecise depth. Therefore, an SIDP approach must be mindful of the expected depth variations in the model's prediction at test time. Accordingly, we introduce an approach that performs continuous modeling of per-pixel depth, where we can predict and reason about the per-pixel depth and its distribution. To this end, we model per-pixel scene depth using a multivariate Gaussian distribution. Moreover, contrary to the existing uncertainty modeling methods -- in the same spirit, where per-pixel depth is assumed to be independent, we introduce per-pixel covariance modeling that encodes its depth dependency w.r.t all the scene points. Unfortunately, per-pixel depth covariance modeling leads to a computationally expensive continuous loss function, which we solve efficiently using the learned low-rank approximation of the overall covariance matrix. Notably, when tested on benchmark datasets such as KITTI, NYU, and SUN-RGB-D, the SIDP model obtained by optimizing our loss function shows state-of-the-art results. Our method's accuracy (named MG) is among the top on the KITTI depth-prediction benchmark leaderboard.

翻译：基于神经网络的单幅图像深度预测（SIDP）是一项具有挑战性的任务，其目标是在测试阶段预测场景中每个像素的深度。由于该问题本质上是不适定的，核心目标在于提出一种能够从训练样本集中可靠建模场景深度的方法。在追求完美深度估计的过程中，现有最先进的多数学习技术会为每个像素预测单个标量深度值。然而，众所周知，训练后的模型存在精度限制，可能预测出不精确的深度。因此，SIDP方法必须在测试时关注模型预测中预期的深度变化。据此，我们提出一种对逐像素深度进行连续建模的方法，可预测并推理逐像素深度及其分布。为此，我们采用多元高斯分布对逐像素场景深度进行建模。此外，与现有相同思路下的不确定性建模方法（假设逐像素深度相互独立）相反，我们引入了逐像素协方差建模，以编码每个像素深度相对于所有场景点的依赖关系。然而，逐像素深度协方差建模会导致计算昂贵的连续损失函数，而本文通过整体协方差矩阵的学习低秩近似有效解决了该问题。值得注意的是，在KITTI、NYU和SUN-RGB-D等基准数据集上的测试结果表明，通过优化我们的损失函数获得的SIDP模型取得了最先进的性能。我们的方法（简称MG）在KITTI深度预测基准排行榜上位居前列。