In this paper, we propose a new theoretical approach to Explainable AI. Following the Scientific Method, this approach consists in formulating on the basis of empirical evidence, a mathematical model to explain and predict the behaviors of Neural Networks. We apply the method to a case study created in a controlled environment, which we call Prime Convolutional Model (p-Conv for short). p-Conv operates on a dataset consisting of the first one million natural numbers and is trained to identify the congruence classes modulo a given integer $m$. Its architecture uses a convolutional-type neural network that contextually processes a sequence of $B$ consecutive numbers to each input. We take an empirical approach and exploit p-Conv to identify the congruence classes of numbers in a validation set using different values for $m$ and $B$. The results show that the different behaviors of p-Conv (i.e., whether it can perform the task or not) can be modeled mathematically in terms of $m$ and $B$. The inferred mathematical model reveals interesting patterns able to explain when and why p-Conv succeeds in performing task and, if not, which error pattern it follows.

翻译：本文提出了一种新的可解释人工智能理论方法。遵循科学方法，该方法基于经验证据构建数学模型，以解释和预测神经网络的行为。我们将此方法应用于在受控环境中创建的案例研究，称之为素数卷积模型（简称p-Conv）。p-Conv操作的数据集包含前一百万个自然数，并训练用于识别给定整数$m$的同余类。其架构采用卷积型神经网络，对每个输入上下文处理$B$个连续数字组成的序列。我们采用实证方法，利用p-Conv在不同$m$和$B$值下识别验证集中数字的同余类。结果表明，p-Conv的不同行为（即能否执行任务）可通过$m$和$B$的数学关系建模。推导出的数学模型揭示了有趣的模式，能够解释p-Conv何时及为何成功执行任务，若失败则遵循何种误差模式。