We revisit the relationship between attention mechanisms and large kernel ConvNets in visual transformers and propose a new spatial attention named Large Kernel Convolutional Attention (LKCA). It simplifies the attention operation by replacing it with a single large kernel convolution. LKCA combines the advantages of convolutional neural networks and visual transformers, possessing a large receptive field, locality, and parameter sharing. We explained the superiority of LKCA from both convolution and attention perspectives, providing equivalent code implementations for each view. Experiments confirm that LKCA implemented from both the convolutional and attention perspectives exhibit equivalent performance. We extensively experimented with the LKCA variant of ViT in both classification and segmentation tasks. The experiments demonstrated that LKCA exhibits competitive performance in visual tasks. Our code will be made publicly available at https://github.com/CatworldLee/LKCA.

翻译：我们重新审视了视觉变换器中注意力机制与大核卷积网络之间的关系，并提出了一种名为大核卷积注意力（LKCA）的新型空间注意力机制。该方法通过用单个大核卷积替换注意力操作来简化计算。LKCA结合了卷积神经网络与视觉变换器的优势，具备大感受野、局部性和参数共享特性。我们从卷积和注意力两个角度解释了LKCA的优越性，并为每种视角提供了等价代码实现。实验证实，从卷积和注意力两个角度实现的LKCA展现出相同的性能。我们在分类和分割任务上对ViT的LKCA变体进行了广泛实验，结果表明LKCA在视觉任务中具有竞争性表现。我们的代码将在 https://github.com/CatworldLee/LKCA 公开。