Dense depth and surface normal predictors should possess the equivariant property to cropping-and-resizing -- cropping the input image should result in cropping the same output image. However, we find that state-of-the-art depth and normal predictors, despite having strong performances, surprisingly do not respect equivariance. The problem exists even when crop-and-resize data augmentation is employed during training. To remedy this, we propose an equivariant regularization technique, consisting of an averaging procedure and a self-consistency loss, to explicitly promote cropping-and-resizing equivariance in depth and normal networks. Our approach can be applied to both CNN and Transformer architectures, does not incur extra cost during testing, and notably improves the supervised and semi-supervised learning performance of dense predictors on Taskonomy tasks. Finally, finetuning with our loss on unlabeled images improves not only equivariance but also accuracy of state-of-the-art depth and normal predictors when evaluated on NYU-v2. GitHub link: https://github.com/mikuhatsune/equivariance

翻译：密集深度与表面法线预测器应具备对裁剪-缩放操作的等变性——对输入图像进行裁剪应导致输出图像被同样裁剪。然而，我们发现最先进的深度与法线预测器尽管性能强劲，却出人意料地不满足等变性。即便在训练中采用裁剪-缩放数据增强，该问题依然存在。为解决此问题，我们提出一种等变正则化技术，包含平均化步骤与自一致性损失，以显式促进深度与法线网络中的裁剪-缩放等变性。我们的方法可应用于CNN与Transformer架构，测试时不增加额外成本，并显著提升Taskonomy任务中密集预测器的监督与半监督学习性能。最后，在未标注图像上使用我们的损失进行微调，不仅能改善等变性，还能提高在NYU-v2上评估的最先进深度与法线预测器的准确性。GitHub链接：https://github.com/mikuhatsune/equivariance