While state-of-the-art monocular depth estimation approaches achieve impressive results in ideal settings, they are highly unreliable under challenging illumination and weather conditions, such as at nighttime or in the presence of rain. In this paper, we uncover these safety-critical issues and tackle them with md4all: a simple and effective solution that works reliably under both adverse and ideal conditions, as well as for different types of learning supervision. We achieve this by exploiting the efficacy of existing methods under perfect settings. Therefore, we provide valid training signals independently of what is in the input. First, we generate a set of complex samples corresponding to the normal training ones. Then, we train the model by guiding its self- or full-supervision by feeding the generated samples and computing the standard losses on the corresponding original images. Doing so enables a single model to recover information across diverse conditions without modifications at inference time. Extensive experiments on two challenging public datasets, namely nuScenes and Oxford RobotCar, demonstrate the effectiveness of our techniques, outperforming prior works by a large margin in both standard and challenging conditions. Source code and data are available at: https://md4all.github.io.
翻译:尽管最先进的单目深度估计方法在理想环境下取得了令人瞩目的结果,但在夜间或雨中等光照恶劣的天气条件下,这些方法高度不可靠。本文揭示了这些安全关键问题,并提出了md4all解决方案:一种简单有效的方法,能在恶劣与理想条件下均可靠运行,并适用于不同类型的学习监督任务。我们通过利用现有方法在完美条件下的有效性来实现这一目标,从而独立于输入内容提供有效的训练信号。首先,我们生成一组与正常训练样本对应的复杂样本;然后,通过输入生成的样本并计算对应原始图像上的标准损失,引导模型进行自监督或全监督训练。这使得单个模型能在无需修改推理过程的情况下,跨不同条件恢复信息。在两个具有挑战性的公开数据集(nuScenes和Oxford RobotCar)上的大量实验表明,我们的技术在标准和挑战性条件下均显著优于先前工作,性能大幅提升。源代码和数据可在 https://md4all.github.io 获取。