We consider applications of neural networks in nonlinear system identification and formulate a hypothesis that adjusting general network structure by incorporating frequency information or other known orthogonal transform, should result in an efficient neural network retaining its universal properties. We show that such a structure is a universal approximator and that using any orthogonal transform in a proposed way implies regularization during training by adjusting the learning rate of each parameter individually. We empirically show in particular, that such a structure, using the Fourier transform, outperforms equivalent models without orthogonality support.

翻译：我们考虑神经网络在非线性系统辨识中的应用，并提出一个假设：通过引入频率信息或其他已知的正交变换来调整网络的一般结构，应当能够产生一个保留其通用特性的高效神经网络。我们证明这种结构具有通用逼近性质，并且以所提出的方式使用任何正交变换，都能通过单独调整每个参数的学习率来实现训练过程中的正则化。我们特别通过实验证明，采用傅里叶变换的这种结构，其性能优于不支持正交性的等效模型。