Photometric constraint is indispensable for self-supervised monocular depth estimation. It involves warping a source image onto a target view using estimated depth&pose, and then minimizing the difference between the warped and target images. However, the endoscopic built-in light causes significant brightness fluctuations, and thus makes the photometric constraint unreliable. Previous efforts only mitigate this relying on extra models to calibrate image brightness. In this paper, we propose MonoPCC to address the brightness inconsistency radically by reshaping the photometric constraint into a cycle form. Instead of only warping the source image, MonoPCC constructs a closed loop consisting of two opposite forward-backward warping paths: from target to source and then back to target. Thus, the target image finally receives an image cycle-warped from itself, which naturally makes the constraint invariant to brightness changes. Moreover, MonoPCC transplants the source image's phase-frequency into the intermediate warped image to avoid structure lost, and also stabilizes the training via an exponential moving average (EMA) strategy to avoid frequent changes in the forward warping. The comprehensive and extensive experimental results on three datasets demonstrate that our proposed MonoPCC shows a great robustness to the brightness inconsistency, and exceeds other state-of-the-arts by reducing the absolute relative error by at least 7.27%.
翻译:光度约束是自监督单目深度估计中不可或缺的部分。该方法利用估计的深度与位姿将源图像扭曲至目标视角,并最小化扭曲图像与目标图像之间的差异。然而,内窥镜内置光源会导致显著的亮度波动,从而使光度约束不可靠。以往研究仅依赖额外模型校准图像亮度来缓解此问题。本文提出MonoPCC,通过将光度约束重塑为循环形式,从根本上解决亮度不一致问题。MonoPCC并非仅扭曲源图像,而是构建由两个相反的前后向扭曲路径组成的闭环:从目标到源,再返回目标。因此,目标图像最终接收到由自身经过循环扭曲的图像,这使得约束天然对亮度变化具有不变性。此外,MonoPCC将源图像的相位频率移植到中间扭曲图像中以避免结构丢失,并通过指数移动平均(EMA)策略稳定训练过程,避免前向扭曲的频繁变化。在三个数据集上的全面实验结果表明,所提出的MonoPCC对亮度不一致性展现出强大的鲁棒性,并将绝对相对误差至少降低7.27%,优于其他现有方法。