Can we recover the hidden parameters of an Artificial Neural Network (ANN) by probing its input-output mapping? We propose a systematic method, called `Expand-and-Cluster' that needs only the number of hidden layers and the activation function of the probed ANN to identify all network parameters. In the expansion phase, we train a series of networks of increasing size using the probed data of the ANN as a teacher. Expansion stops when a minimal loss is consistently reached in networks of a given size. In the clustering phase, weight vectors of the expanded students are clustered, which allows structured pruning of superfluous neurons in a principled way. We find that an overparameterization of a factor four is sufficient to reliably identify the minimal number of neurons and to retrieve the original network parameters in $80\%$ of tasks across a family of 150 toy problems of variable difficulty. Furthermore, shallow and deep teacher networks trained on MNIST data can be identified with less than $5\%$ overhead in the neuron number. Thus, while direct training of a student network with a size identical to that of the teacher is practically impossible because of the highly non-convex loss function, training with mild overparameterization followed by clustering and structured pruning correctly identifies the target network.

翻译：能否通过探测人工神经网络（ANN）的输入-输出映射来恢复其隐藏参数？我们提出一种名为“扩展-聚类”的系统方法，该方法仅需探测ANN的隐藏层数量和激活函数便可识别所有网络参数。在扩展阶段，我们利用探测到的ANN数据作为教师，训练一系列规模递增的网络。当在特定规模的网络中持续达到最小损失时，扩展过程终止。在聚类阶段，对扩展后学生网络的权重向量进行聚类，从而以结构化方式对冗余神经元进行有原则的剪枝。研究发现，四倍过参数化足以可靠地识别最小神经元数量，并在包含150个难度各异玩具问题的任务族中，以80%的成功率恢复原始网络参数。此外，在MNIST数据上训练的浅层和深层教师网络可在神经元数量开销低于5%的情况下被识别。因此，尽管由于高度非凸的损失函数，直接训练与教师网络规模相同的网络实际不可行，但采用适度过参数化训练后结合聚类与结构化剪枝的方法，能够正确识别目标网络。