In this paper an attractor FCM is created, tested, and analyzed. This FCM is neither a hebbian based nor agentic, nor a hybrid; it rather is a gradient descent based, physics constrained, Jacobian version of an FCM. Moreover, this model has several quirks; it uses residual memory, back propagation through time, and a fixed point anchor that is recursively implemented to update its weights. The residuals update the recursive part without losing the system memory. The model's anchor enables it to converge in a fixed point for which back propagation through time unrolls it and ensures that the error minimization is for an accurate gradient. Furthermore, a new learning algorithm is utilized. The Newton's method finds the system's fixed point attractor and then gradient descend is adaptively changing the landscape; an adaptive term is used to directly manipulate the weights through the attractor dynamics. As the adaptive term changes, the descent through the landscape is constantly adjusting according to sigmoid saturation, and that prevents premature convergence to a local minimum. Lastly, the updates are filtered by causal mask that informs the network about the physics, respecting the initial expert based opinions, for which model reduces the error to the target in an efficient way.
翻译:本文创建、测试并分析了一种吸引子FCM。该FCM既非基于赫布学习,也非基于智能体或混合模型,而是一种基于梯度下降、受物理约束的雅可比版本FCM。此外,该模型具有若干特点:它采用残差记忆、随时间反向传播以及递归实现的定点锚来更新权重。残差在更新递归部分时不丢失系统记忆。模型的锚使其能够收敛到定点,而随时间反向传播展开该定点,确保误差最小化针对精确梯度。同时,采用了一种新的学习算法。牛顿法寻找系统的定点吸引子,随后梯度下降自适应地改变损失曲面;一个自适应项通过吸引子动力学直接操纵权重。随着自适应项的变化,损失曲面上的下降过程根据sigmoid饱和程度不断调整,从而防止过早收敛到局部最小值。最后,更新被因果掩码过滤,该掩码告知网络物理约束,并尊重初始专家意见,使模型高效地将误差降至目标值。