Most invariance-based self-supervised methods rely on single object-centric images (e.g., ImageNet images) for pretraining, learning invariant representations from geometric transformations. However, when images are not object-centric, the semantics of the image can be significantly altered due to geometric transformations such as random crops and multi-crops. Furthermore, the model may struggle to capture location information. For this reason, we propose a Geometric Transformation Sensitive Architecture that learns features sensitive to geometric transformation like four-fold rotation, random crop, and multi-crop. Our method encourages the student to learn sensitive features by increasing the similarity between overlapping regions not entire views. and applying rotations to the target feature map. Additionally, we use a patch correspondence loss to capture long-term dependencies. Our approach demonstrates improved performance when using non-object-centric images as pretraining data compared to other methods that learn geometric transformation-invariant representations. We surpass DINO baseline in tasks such as image classification, semantic segmentation, detection, and instance segmentation with improvements of 6.1 $Acc$, 0.6 $mIoU$, 0.4 $AP^b$, and 0.1 $AP^m$.

翻译：大多数基于不变性的自监督方法依赖单物体中心图像（如ImageNet图像）进行预训练，通过几何变换学习不变表征。然而，当图像并非以物体为中心时，随机裁剪和多裁剪等几何变换可能显著改变图像语义。此外，模型可能难以捕捉位置信息。为此，我们提出一种几何变换敏感架构，用于学习对四折旋转、随机裁剪和多裁剪等几何变换敏感的特征。该方法通过增大重叠区域而非整个视图的相似度，并对目标特征图施加旋转操作，促使学生模型学习敏感特征。同时，我们采用图像块对应损失以捕获长程依赖关系。实验表明，相比其他学习几何变换不变表征的方法，本方法在使用非物体中心图像作为预训练数据时性能更优。在图像分类、语义分割、目标检测和实例分割任务中，我们分别以6.1 $Acc$、0.6 $mIoU$、0.4 $AP^b$和0.1 $AP^m$的提升幅度超越了DINO基线。