The paper proposes a new algorithm called SymBa that aims to achieve more biologically plausible learning than Back-Propagation (BP). The algorithm is based on the Forward-Forward (FF) algorithm, which is a BP-free method for training neural networks. SymBa improves the FF algorithm's convergence behavior by addressing the problem of asymmetric gradients caused by conflicting converging directions for positive and negative samples. The algorithm balances positive and negative losses to enhance performance and convergence speed. Furthermore, it modifies the FF algorithm by adding Intrinsic Class Pattern (ICP) containing class information to prevent the loss of class information during training. The proposed algorithm has the potential to improve our understanding of how the brain learns and processes information and to develop more effective and efficient artificial intelligence systems. The paper presents experimental results that demonstrate the effectiveness of SymBa algorithm compared to the FF algorithm and BP.
翻译:本文提出一种名为SymBa的新算法,旨在实现比反向传播(BP)更具生物合理性的学习机制。该算法基于前向-前向(FF)算法——一种无需反向传播的神经网络训练方法。SymBa通过解决正负样本收敛方向冲突导致的非对称梯度问题,改进了FF算法的收敛行为。算法通过平衡正负损失以提升性能与收敛速度,并进一步修改FF算法:通过添加包含类别信息的内部类别模式(ICP)来防止训练过程中类别信息丢失。所提算法有望增进对大脑学习与信息处理机制的理解,同时推动更高效人工智能系统的发展。实验结果表明,SymBa算法在有效性上优于FF算法与BP算法。