Photometric constraint is indispensable for self-supervised monocular depth estimation. It involves warping a source image onto a target view using estimated depth&pose, and then minimizing the difference between the warped and target images. However, the endoscopic built-in light causes significant brightness fluctuations, and thus makes the photometric constraint unreliable. Previous efforts only mitigate this relying on extra models to calibrate image brightness. In this paper, we propose MonoPCC to address the brightness inconsistency radically by reshaping the photometric constraint into a cycle form. Instead of only warping the source image, MonoPCC constructs a closed loop consisting of two opposite forward-backward warping paths: from target to source and then back to target. Thus, the target image finally receives an image cycle-warped from itself, which naturally makes the constraint invariant to brightness changes. Moreover, MonoPCC transplants the source image's phase-frequency into the intermediate warped image to avoid structure lost, and also stabilizes the training via an exponential moving average (EMA) strategy to avoid frequent changes in the forward warping. The comprehensive and extensive experimental results on four endoscopic datasets demonstrate that our proposed MonoPCC shows a great robustness to the brightness inconsistency, and exceeds other state-of-the-arts by reducing the absolute relative error by at least 7.27%, 9.38%, 9.90% and 3.17%, respectively.
翻译:光度约束是自监督单目深度估计中不可或缺的组成部分。该方法利用估计的深度与位姿将源图像扭曲至目标视角,并最小化扭曲图像与目标图像之间的差异。然而,内窥镜内置光源会导致显著的亮度波动,从而使光度约束不可靠。以往的研究仅依赖额外模型校准图像亮度来缓解这一问题。本文提出MonoPCC,通过将光度约束重塑为循环形式,从根本上解决了亮度不一致性问题。MonoPCC并非仅对源图像进行扭曲,而是构建了一个由两个相反的前向-后向扭曲路径组成的闭环:从目标到源,再回到目标。因此,目标图像最终接收到的图像是由自身循环扭曲而来,这使得约束自然地对亮度变化具有不变性。此外,MonoPCC将源图像的相位频率移植至中间扭曲图像以避免结构信息丢失,并通过指数移动平均(EMA)策略稳定训练过程,避免前向扭曲中的频繁变化。在四个内窥镜数据集上的全面且广泛的实验结果表明,所提出的MonoPCC对亮度不一致性具有极强的鲁棒性,并在绝对相对误差上分别降低了至少7.27%、9.38%、9.90%和3.17%,超越了其他最先进方法。