Most existing methods often rely on complex models to predict scene depth with high accuracy, resulting in slow inference that is not conducive to deployment. To better balance precision and speed, we first designed SmallDepth based on sparsity. Second, to enhance the feature representation ability of SmallDepth during training under the condition of equal complexity during inference, we propose an equivalent transformation module(ETM). Third, to improve the ability of each layer in the case of a fixed SmallDepth to perceive different context information and improve the robustness of SmallDepth to the left-right direction and illumination changes, we propose pyramid loss. Fourth, to further improve the accuracy of SmallDepth, we utilized the proposed function approximation loss (APX) to transfer knowledge in the pretrained HQDecv2, obtained by optimizing the previous HQDec to address grid artifacts in some regions, to SmallDepth. Extensive experiments demonstrate that each proposed component improves the precision of SmallDepth without changing the complexity of SmallDepth during inference, and the developed approach achieves state-of-the-art results on KITTI at an inference speed of more than 500 frames per second and with approximately 2 M parameters. The code and models will be publicly available at https://github.com/fwucas/FA-Depth.
翻译:现有方法大多依赖复杂模型以实现高精度场景深度预测,导致推理速度缓慢,不利于实际部署。为更好地平衡精度与速度,我们首先基于稀疏性设计了SmallDepth;其次,为在推理复杂度不变条件下增强SmallDepth训练时的特征表征能力,提出等效变换模块(ETM);第三,为在固定SmallDepth结构下提升各层感知不同上下文信息的能力,并增强其对左右方向及光照变化的鲁棒性,提出金字塔损失函数;第四,为进一步提高SmallDepth精度,利用所提出的函数近似损失(APX)将预训练HQDecv2中的知识迁移至SmallDepth(该预训练模型通过对先前HQDec进行优化以解决部分区域网格伪影问题)。大量实验表明,各提出组件在不改变SmallDepth推理复杂度的情况下均能提升其精度,所开发方法在KITTI数据集上以约200万参数实现超过500帧/秒的推理速度并达到最先进水平。代码与模型将发布于https://github.com/fwucas/FA-Depth。