In this paper an attractor FCM is created, tested, and analyzed. This FCM is neither a hebbian based nor agentic, nor a hybrid; it rather is a gradient descent based, physics constrained, Jacobian version of an FCM. Moreover, this model has several quirks; it uses residual memory, back propagation through time, and a fixed point anchor that is recursively implemented to update its weights. The residuals update the recursive part without losing the system memory. The model's anchor enables it to converge in a fixed point for which back propagation through time unrolls it and ensures that the error minimization is for an accurate gradient. Furthermore, a new learning algorithm is utilized. The Newton's method finds the system's fixed point attractor and then gradient descend is adaptively changing the landscape; an adaptive term is used to directly manipulate the weights through the attractor dynamics. As the adaptive term changes, the descent through the landscape is constantly adjusting according to sigmoid saturation, and that prevents premature convergence to a local minimum. Lastly, the updates are filtered by causal mask that informs the network about the physics, respecting the initial expert based opinions, for which model reduces the error to the target in an efficient way.
翻译:本文构建、测试并分析了吸引子FCM。该FCM既非基于赫布学习,也非代理型或混合型,而是基于梯度下降、受物理约束的雅可比版本FCM。此外,该模型具有若干特性:它采用残差记忆、时间反向传播以及递归实现的定点锚来更新权重。残差在更新递归部分时不丢失系统记忆。模型的锚点使其能够收敛至一个定点,时间反向传播通过展开该定点确保误差最小化获得精确梯度。同时,模型采用了一种新的学习算法:牛顿法找到系统的定点吸引子,随后梯度下降自适应地改变势能面;通过自适应项直接操控吸引子动态下的权重。随着自适应项的变化,势能面上的下降过程根据sigmoid饱和程度持续调整,从而防止过早收敛至局部极小值。最后,更新操作通过因果掩码进行过滤,该掩码向网络传递物理约束,保留初始专家先验知识,使模型能够高效地将误差降至目标值。