The remarkable success of deep neural networks (DNN) is often attributed to their high expressive power and their ability to approximate functions of arbitrary complexity. Indeed, DNNs are highly non-linear models, and activation functions introduced into them are largely responsible for this. While many works studied the expressive power of DNNs through the lens of their approximation capabilities, quantifying the non-linearity of DNNs or of individual activation functions remains an open problem. In this paper, we propose the first theoretically sound solution to track non-linearity propagation in deep neural networks with a specific focus on computer vision applications. Our proposed affinity score allows us to gain insights into the inner workings of a wide range of different architectures and learning paradigms. We provide extensive experimental results that highlight the practical utility of the proposed affinity score and its potential for long-reaching applications.

翻译：深度神经网络（DNN）的巨大成功通常归因于其强大的表达能力以及逼近任意复杂度函数的能力。事实上，DNN是高度非线性的模型，其中引入的激活函数对此起到了关键作用。尽管已有大量研究通过逼近能力视角探讨DNN的表达能力，但量化DNN或单个激活函数的非线性仍然是一个悬而未决的问题。本文针对计算机视觉应用，首次提出了一种理论上严谨的方法来追踪深度神经网络中的非线性传播。我们提出的亲和度得分能够揭示多种不同架构与学习范式的内部运行机理。通过广泛的实验结果，我们展示了该亲和度得分的实用价值及其在长远应用中的潜力。