Predicting accurate depth with monocular images is important for low-cost robotic applications and autonomous driving. This study proposes a comprehensive self-supervised framework for accurate scale-aware depth prediction on autonomous driving scenes utilizing inter-frame poses obtained from inertial measurements. In particular, we introduce a Full-Scale depth prediction network named FSNet. FSNet contains four important improvements over existing self-supervised models: (1) a multichannel output representation for stable training of depth prediction in driving scenarios, (2) an optical-flow-based mask designed for dynamic object removal, (3) a self-distillation training strategy to augment the training process, and (4) an optimization-based post-processing algorithm in test time, fusing the results from visual odometry. With this framework, robots and vehicles with only one well-calibrated camera can collect sequences of training image frames and camera poses, and infer accurate 3D depths of the environment without extra labeling work or 3D data. Extensive experiments on the KITTI dataset, KITTI-360 dataset and the nuScenes dataset demonstrate the potential of FSNet. More visualizations are presented in \url{https://sites.google.com/view/fsnet/home}
翻译:使用单目图像预测精确深度对于低成本机器人应用和自动驾驶具有重要意义。本研究提出了一种综合性的自监督框架,利用惯性测量获得的帧间位姿,实现自动驾驶场景中精确的尺度感知深度预测。具体而言,我们引入了一个名为FSNet的全尺度深度预测网络。与现有自监督模型相比,FSNet包含四项重要改进:(1) 面向驾驶场景深度预测稳定训练的多通道输出表示;(2) 基于光流法的动态目标去除掩模;(3) 增强训练过程的自蒸馏训练策略;(4) 融合视觉里程计结果的测试时优化后处理算法。借助该框架,仅配备单台精确标定相机的机器人和车辆即可采集训练图像帧序列及相机位姿,无需额外标注工作或三维数据即可推断环境精确的三维深度。在KITTI数据集、KITTI-360数据集和nuScenes数据集上的大量实验证明了FSNet的潜力。更多可视化结果见\url{https://sites.google.com/view/fsnet/home}。