Recent approximations to backpropagation (BP) have mitigated many of BP's computational inefficiencies and incompatibilities with biology, but important limitations still remain. Moreover, the approximations significantly decrease accuracy in benchmarks, suggesting that an entirely different approach may be more fruitful. Here, grounded on recent theory for Hebbian learning in soft winner-take-all networks, we present multilayer SoftHebb, i.e. an algorithm that trains deep neural networks, without any feedback, target, or error signals. As a result, it achieves efficiency by avoiding weight transport, non-local plasticity, time-locking of layer updates, iterative equilibria, and (self-) supervisory or other feedback signals -- which were necessary in other approaches. Its increased efficiency and biological compatibility do not trade off accuracy compared to state-of-the-art bio-plausible learning, but rather improve it. With up to five hidden layers and an added linear classifier, accuracies on MNIST, CIFAR-10, STL-10, and ImageNet, respectively reach 99.4%, 80.3%, 76.2%, and 27.3%. In conclusion, SoftHebb shows with a radically different approach from BP that Deep Learning over few layers may be plausible in the brain and increases the accuracy of bio-plausible machine learning. Code is available at https://github.com/NeuromorphicComputing/SoftHebb.

翻译：近期对反向传播（BP）的近似方法已缓解了BP的诸多计算低效及与生物学不兼容的问题，但仍存在重要局限。此外，这些近似方法显著降低了基准测试的精度，表明采用完全不同的方法可能更具前景。基于软赢家通吃网络中赫布学习的最新理论，本文提出多层SoftHebb算法——该算法无需任何反馈、目标或误差信号即可训练深度神经网络。因此，它通过避免权重传输、非局部可塑性、层更新时间锁定、迭代均衡以及（自）监督或其他反馈信号实现高效性（这些是其他方法所必需的）。相较于当前最先进的生物可解释学习方法，其更高效率与生物兼容性不仅未以精度为代价，反而提升了精度。在包含最多五个隐藏层及附加线性分类器的情况下，该方法在MNIST、CIFAR-10、STL-10和ImageNet上的准确率分别达到99.4%、80.3%、76.2%和27.3%。结论表明，SoftHebb以与BP截然不同的方法证明了少层深度学习在大脑中的可行性，并提高了生物可解释机器学习的精度。代码开源于https://github.com/NeuromorphicComputing/SoftHebb。