Humans and animals recognize objects irrespective of the beholder's point of view, which may drastically change their appearances. Artificial pattern recognizers also strive to achieve this, e.g., through translational invariance in convolutional neural networks (CNNs). However, both CNNs and vision transformers (ViTs) perform very poorly on rotated inputs. Here we present artificial mental rotation (AMR), a novel deep learning paradigm for dealing with in-plane rotations inspired by the neuro-psychological concept of mental rotation. Our simple AMR implementation works with all common CNN and ViT architectures. We test it on ImageNet, Stanford Cars, and Oxford Pet. With a top-1 error (averaged across datasets and architectures) of $0.743$, AMR outperforms the current state of the art (rotational data augmentation, average top-1 error of $0.626$) by $19\%$. We also easily transfer a trained AMR module to a downstream task to improve the performance of a pre-trained semantic segmentation model on rotated CoCo from $32.7$ to $55.2$ IoU.

翻译：人类和动物能够识别物体，无论观察者的视角如何变化，即使这些变化会显著改变物体的外观。人工模式识别系统也致力于实现这一目标，例如通过卷积神经网络（CNNs）中的平移不变性。然而，无论是CNNs还是视觉变换器（ViTs），在处理旋转输入时表现都非常差。本文提出人工心理旋转（AMR），这是一种受神经心理学中心理旋转概念启发的新型深度学习范式，用于处理平面内旋转。我们的简单AMR实现适用于所有常见的CNN和ViT架构。我们在ImageNet、Stanford Cars和Oxford Pet数据集上进行了测试。AMR的top-1错误率（跨数据集和架构的平均值）为0.743，比当前最先进的方法（旋转数据增强，平均top-1错误率为0.626）提升了19%。我们还可以轻松地将训练好的AMR模块迁移到下游任务中，使预训练的语义分割模型在旋转CoCo数据集上的IoU从32.7提升到55.2。