Neural networks are omnipresent, but remain poorly understood. Their increasing complexity and use in critical systems raises the important challenge to full interpretability. We propose to address a simple well-posed learning problem: estimating the radius of a centred pulse in a one-dimensional signal or of a centred disk in two-dimensional images using a simple convolutional neural network. Surprisingly, understanding what trained networks have learned is difficult and, to some extent, counter-intuitive. However, an in-depth theoretical analysis in the one-dimensional case allows us to comprehend constraints due to the chosen architecture, the role of each filter and of the nonlinear activation function, and every single value taken by the weights of the model. Two fundamental concepts of neural networks arise: the importance of invariance and of the shape of the nonlinear activation functions.

翻译：神经网络无处不在，但其工作机制仍未被充分理解。随着其复杂性的增加及在关键系统中的广泛应用，实现完全可解释性成为一项重要挑战。本文针对一个简单且适定的学习问题展开研究：使用简单卷积神经网络估计一维信号中中心脉冲的半径，或二维图像中中心圆盘的半径。令人惊讶的是，理解训练后网络的学习内容具有相当难度，甚至存在反直觉之处。然而，通过一维情况下的深度理论分析，我们能够揭示所选架构带来的约束、每个滤波器及非线性激活函数的作用，以及模型中每个权重值的具体含义。神经网络的两个基本概念由此浮现：不变性的重要性以及非线性激活函数形状的关键作用。