The phenomena of Spectral Bias, where the higher frequency components of a function being learnt in a feedforward Artificial Neural Network (ANN) are seen to converge more slowly than the lower frequencies, is observed ubiquitously across ANNs. This has created technology challenges in fields where resolution of higher frequencies is crucial, like in Physics Informed Neural Networks (PINNs). Extreme Learning Machines (ELMs) that obviate an iterative solution process which provides the theoretical basis of Spectral Bias (SB), should in principle be free of the same. This work verifies the reliability of this assumption, and shows that it is incorrect. However, the structure of ELMs makes them naturally amenable to implementation of variants of Fourier Feature Embeddings, which have been shown to mitigate SB in ANNs. This approach is implemented and verified to completely eliminate SB, thus bringing into feasibility the application of ELMs for practical problems like PINNs where resolution of higher frequencies is essential.
翻译:谱偏差现象是指在前馈人工神经网络中,函数的高频分量收敛速度明显慢于低频分量的现象,该现象在各类人工神经网络中普遍存在。这给需要高频分辨率的关键领域(如物理信息神经网络)带来了技术挑战。极端学习机避免了提供谱偏差理论基础的迭代求解过程,理论上应不受此影响。本研究验证了该假设的可靠性,并证明其并不成立。然而,极端学习机的结构使其天然适合实现多种傅里叶特征嵌入变体,这些变体已被证明可缓解人工神经网络中的谱偏差问题。本文实现了这一方法并验证其能完全消除谱偏差,从而使得极端学习机能够应用于需要高频分辨率的实际工程问题,如物理信息神经网络。