Understanding the learning process of artificial neural networks requires clarifying the structure of the parameter space within which learning takes place. A neural network parameter's functional equivalence class is the set of parameters implementing the same input--output function. For many architectures, almost all parameters have a simple and well-documented functional equivalence class. However, there is also a vanishing minority of reducible parameters, with richer functional equivalence classes caused by redundancies among the network's units. In this paper, we give an algorithmic characterisation of unit redundancies and reducible functional equivalence classes for a single-hidden-layer hyperbolic tangent architecture. We show that such functional equivalence classes are piecewise-linear path-connected sets, and that for parameters with a majority of redundant units, the sets have a diameter of at most 7 linear segments.

翻译：理解人工神經網絡的學習過程需要闡明學習發生的參數空間結構。神經網絡參數的功能等價類是指實現相同輸入-輸出函數的參數集合。對於許多架構而言，幾乎所有參數都具有簡單且記錄完備的功能等價類。然而，也存在極少數可約參數，由於網絡單元之間的冗餘性，其功能等價類更為豐富。本文針對單隱藏層雙曲正切架構，給出了單元冗餘性與可約功能等價類的算法表徵。我們證明此類功能等價類是分段線性路徑連通集，且對於具有多數冗餘單元的參數，這些集合的直徑至多包含7個線性段。