Random functional-linked types of neural networks (RFLNNs), e.g., the extreme learning machine (ELM) and broad learning system (BLS), which avoid suffering from a time-consuming training process, offer an alternative way of learning in deep structure. The RFLNNs have achieved excellent performance in various classification and regression tasks, however, the properties and explanations of these networks are ignored in previous research. This paper gives some insights into the properties of RFLNNs from the viewpoints of frequency domain, and discovers the presence of frequency principle in these networks, that is, they preferentially capture low-frequencies quickly and then fit the high frequency components during the training process. These findings are valuable for understanding the RFLNNs and expanding their applications. Guided by the frequency principle, we propose a method to generate a BLS network with better performance, and design an efficient algorithm for solving Poison's equation in view of the different frequency principle presenting in the Jacobi iterative method and BLS network.
翻译:随机函数链接型神经网络(RFLNNs),例如极限学习机(ELM)和宽度学习系统(BLS),通过避免耗时的训练过程,为深度学习提供了一种替代方案。尽管RFLNNs在各种分类与回归任务中取得了优异性能,但以往研究对其网络属性与解释有所忽略。本文从频域视角揭示了RFLNNs的若干属性,并发现这些网络中存在频率原理——即在训练过程中,它们优先快速捕获低频成分,随后拟合高频成分。这些发现对于理解RFLNNs并拓展其应用具有重要价值。基于频率原理的指导,我们提出了一种生成性能更优的BLS网络的方法,并针对Jacobi迭代法与BLS网络中呈现的不同频率原理,设计了一种高效求解泊松方程的算法。