Activity difference based learning algorithms-such as contrastive Hebbian learning and equilibrium propagation-have been proposed as biologically plausible alternatives to error back-propagation. However, on traditional digital chips these algorithms suffer from having to solve a costly inference problem twice, making these approaches more than two orders of magnitude slower than back-propagation. In the analog realm equilibrium propagation may be promising for fast and energy efficient learning, but states still need to be inferred and stored twice. Inspired by lifted neural networks and compartmental neuron models we propose a simple energy based compartmental neuron model, termed dual propagation, in which each neuron is a dyad with two intrinsic states. At inference time these intrinsic states encode the error/activity duality through their difference and their mean respectively. The advantage of this method is that only a single inference phase is needed and that inference can be solved in layerwise closed-form. Experimentally we show on common computer vision datasets, including Imagenet32x32, that dual propagation performs equivalently to back-propagation both in terms of accuracy and runtime.

翻译：基于活动差异的学习算法——如对比赫布学习（contrastive Hebbian learning）和平衡传播（equilibrium propagation）——已被提出作为误差反向传播的生物学可行替代方案。然而，在传统数字芯片上，这些算法需要两次求解成本高昂的推理问题，导致其速度比反向传播慢两个数量级以上。在模拟领域中，平衡传播有望实现快速且节能的学习，但状态仍需两次推断和存储。受提升神经网络（lifted neural networks）和区室神经元模型的启发，我们提出了一种简单的基于能量的区室神经元模型，称为双重传播（dual propagation），其中每个神经元是一个具有两种内在状态的二元体。在推理时，这些内在状态分别通过其差值和均值来编码误差/活动二元性。该方法的优势在于只需一次推理阶段，且推理可通过逐层闭式求解。实验表明，在包括ImageNet32x32在内的常见计算机视觉数据集上，双重传播在准确性和运行时间方面均与反向传播表现相当。