Convolutional Neural Networks (CNNs) have been heavily used in Deep Learning due to their success in various tasks. Nonetheless, it has been observed that CNNs suffer from redundancy in feature maps, leading to inefficient capacity utilization. Efforts to mitigate and solve this problem led to the emergence of multiple methods, amongst which is kernel orthogonality through variant means. In this work, we challenge the common belief that kernel orthogonality leads to a decrease in feature map redundancy, which is, supposedly, the ultimate objective behind kernel orthogonality. We prove, theoretically and empirically, that kernel orthogonality has an unpredictable effect on feature map similarity and does not necessarily decrease it. Based on our theoretical result, we propose an effective method to reduce feature map similarity independently of the input of the CNN. This is done by minimizing a novel loss function we call Convolutional Similarity. Empirical results show that minimizing the Convolutional Similarity increases the performance of classification models and can accelerate their convergence. Furthermore, using our proposed method pushes towards a more efficient use of the capacity of models, allowing the use of significantly smaller models to achieve the same levels of performance.

翻译：卷积神经网络（CNN）因其在各类任务中的卓越表现而被广泛应用于深度学习领域。然而，现有研究发现CNN存在特征图冗余问题，导致模型容量利用率低下。为缓解和解决该问题，学界提出了多种方法，其中包含通过不同途径实现卷积核正交化的技术。本研究对"卷积核正交性能降低特征图冗余性"这一普遍认知提出质疑——该观点通常被认为是追求卷积核正交性的根本目标。我们通过理论证明与实验验证表明：卷积核正交性对特征图相似度的影响具有不可预测性，且未必会降低特征图相似度。基于理论分析结果，我们提出了一种独立于CNN输入数据的特征图相似度降低方法，该方法通过最小化我们提出的新型损失函数——卷积相似度损失来实现。实验结果表明：最小化卷积相似度能够提升分类模型的性能并加速其收敛过程。此外，采用本方法可促进模型容量的高效利用，使得采用显著更小的模型即可达到同等性能水平。