Accurate and efficient dense metric depth estimation is crucial for 3D visual perception in robotics and XR. In this paper, we develop a monocular visual-inertial motion and depth (VIMD) learning framework to estimate dense metric depth by leveraging accurate and efficient MSCKF-based monocular visual-inertial motion tracking. At the core the proposed VIMD is to exploit multi-view information to iteratively refine per-pixel scale, instead of globally fitting an invariant affine model as in the prior work. The VIMD framework is highly modular, making it compatible with a variety of existing depth estimation backbones. We conduct extensive evaluations on the TartanAir and VOID datasets and demonstrate its zero-shot generalization capabilities on the AR Table dataset. Our results show that VIMD achieves exceptional accuracy and robustness, even with extremely sparse points as few as 10-20 metric depth points per image. This makes the proposed VIMD a practical solution for deployment in resource constrained settings, while its robust performance and strong generalization capabilities offer significant potential across a wide range of scenarios.

翻译：精确且高效的稠密度量深度估计对于机器人和XR中的三维视觉感知至关重要。本文提出了一种单目视觉惯性运动与深度学习框架，通过利用基于MSCKF的精确高效单目视觉惯性运动跟踪来估计稠密度量深度。所提出的VIMD方法的核心在于利用多视图信息迭代优化每个像素的尺度，而非如先前工作那样全局拟合一个不变仿射模型。VIMD框架具有高度模块化特性，使其能够与多种现有深度估计主干网络兼容。我们在TartanAir和VOID数据集上进行了广泛评估，并在AR Table数据集上展示了其零样本泛化能力。实验结果表明，即使每幅图像仅使用极稀疏的10-20个度量深度点，VIMD仍能实现卓越的精度与鲁棒性。这使得所提出的VIMD成为资源受限环境下部署的实用解决方案，其稳健的性能和强大的泛化能力在广泛的应用场景中展现出巨大潜力。