The forward-forward algorithm presents a new method of training neural networks by updating weights during an inference, performing parameter updates for each layer individually. This immediately reduces memory requirements during training and may lead to many more benefits, like seamless online training. This method relies on a loss ("goodness") function that can be evaluated on the activations of each layer, of which can have a varied parameter size, depending on the hyperparamaterization of the network. In the seminal paper, a goodness function was proposed to fill this need; however, if placed in a one-class problem context, one need not pioneer a new loss because these functions can innately handle dynamic network sizes. In this paper, we investigate the performance of deep one-class objective functions when trained in a forward-forward fashion. The code is available at \url{https://github.com/MichaelHopwood/ForwardForwardOneclass}.
翻译:前向-前向算法提出了一种在推理过程中更新权重的神经网络训练新方法,该方法对每一层的参数进行独立更新。这种方式不仅直接减少了训练过程中的内存需求,还可能带来诸多益处,例如实现无缝在线训练。该算法依赖于一种"优良度"损失函数,该函数可对每一层的激活值进行评估,而各层的参数规模则取决于网络的超参数设置。在开创性论文中,研究者提出了一种满足该需求的优良度函数;然而,若将其置于单类问题背景下,则无需重新设计损失函数——因为这类函数本身就能适应动态变化的网络规模。本文系统研究了深度单类目标函数在前向-前向训练范式下的性能表现。相关代码可通过 \url{https://github.com/MichaelHopwood/ForwardForwardOneclass} 获取。