Can we recover the hidden parameters of an Artificial Neural Network (ANN) by probing its input-output mapping? We propose a systematic method, called `Expand-and-Cluster' that needs only the number of hidden layers and the activation function of the probed ANN to identify all network parameters. In the expansion phase, we train a series of student networks of increasing size using the probed data of the ANN as a teacher. Expansion stops when a minimal loss is consistently reached in student networks of a given size. In the clustering phase, weight vectors of the expanded students are clustered, which allows structured pruning of superfluous neurons in a principled way. We find that an overparameterization of a factor four is sufficient to reliably identify the minimal number of neurons and to retrieve the original network parameters in $80\%$ of tasks across a family of 150 toy problems of variable difficulty. Furthermore, a teacher network trained on MNIST data can be identified with less than $5\%$ overhead in the neuron number. Thus, while direct training of a student network with a size identical to that of the teacher is practically impossible because of the non-convex loss function, training with mild overparameterization followed by clustering and structured pruning correctly identifies the target network.

翻译：我们能否通过探测人工神经网络（ANN）的输入-输出映射来恢复其隐藏参数？我们提出了一种名为“扩展与聚类”（Expand-and-Cluster）的系统方法，该方法仅需探测ANN的隐藏层数量和激活函数即可识别所有网络参数。在扩展阶段，我们利用ANN的探测数据作为教师，训练一系列规模递增的学生网络。当给定规模的学生网络持续达到最小损失时，扩展阶段停止。在聚类阶段，对扩展后学生网络的权重向量进行聚类，从而以原理性方式对多余神经元进行结构化剪枝。我们发现，过参数化因子达到四倍时，便足以在包含150个不同难度玩具问题的任务族中，以80%的任务成功率可靠地识别最小神经元数量并恢复原始网络参数。此外，在MNIST数据上训练的教师网络，其神经元数量的识别开销可控制在5%以内。因此，尽管由于非凸损失函数，直接训练与教师网络规模相同的学生网络实际上不可行，但通过适度过参数化结合聚类与结构化剪枝，能够正确识别目标网络。