Photometric constraint is indispensable for self-supervised monocular depth estimation. It involves warping a source image onto a target view using estimated depth&pose, and then minimizing the difference between the warped and target images. However, the endoscopic built-in light causes significant brightness fluctuations, and thus makes the photometric constraint unreliable. Previous efforts only mitigate this relying on extra models to calibrate image brightness. In this paper, we propose MonoPCC to address the brightness inconsistency radically by reshaping the photometric constraint into a cycle form. Instead of only warping the source image, MonoPCC constructs a closed loop consisting of two opposite forward-backward warping paths: from target to source and then back to target. Thus, the target image finally receives an image cycle-warped from itself, which naturally makes the constraint invariant to brightness changes. Moreover, MonoPCC transplants the source image's phase-frequency into the intermediate warped image to avoid structure lost, and also stabilizes the training via an exponential moving average (EMA) strategy to avoid frequent changes in the forward warping. The comprehensive and extensive experimental results on four endoscopic datasets demonstrate that our proposed MonoPCC shows a great robustness to the brightness inconsistency, and exceeds other state-of-the-arts by reducing the absolute relative error by at least 7.27%, 9.38%, 9.90% and 3.17%, respectively.
翻译:光度约束在自监督单目深度估计中不可或缺。该方法通过估计的深度与姿态将源图像扭曲至目标视角,进而最小化扭曲图像与目标图像之间的差异。然而,内窥镜内置光源会引起显著的亮度波动,从而导致光度约束失效。先前研究仅依赖额外模型进行图像亮度校正以缓解此问题。本文提出MonoPCC方法,通过将光度约束重构为循环形式,从根本上解决亮度不一致问题。MonoPCC不仅扭曲源图像,还构建了一个包含两条相反前向-反向扭曲路径的闭环:从目标图像到源图像,再返回目标图像。由此,目标图像最终获得一个从自身经循环扭曲的图像,该机制自然使得约束对亮度变化保持不变性。此外,MonoPCC将源图像的相位频率移植至中间扭曲图像以避免结构丢失,并通过指数移动平均(EMA)策略稳定训练过程,防止前向扭曲的频繁变化。在四个内窥镜数据集上的全面实验结果表明,所提出的MonoPCC对亮度不一致具有极强的鲁棒性,其绝对相对误差分别至少降低7.27%、9.38%、9.90%和3.17%,性能超越其他现有最优方法。