Deep learning has redefined the field of artificial intelligence (AI) thanks to the rise of artificial neural networks, which are architectures inspired by their neurological counterpart in the brain. Through the years, this dualism between AI and neuroscience has brought immense benefits to both fields, allowing neural networks to be used in dozens of applications. These networks use an efficient implementation of reverse differentiation, called backpropagation (BP). This algorithm, however, is often criticized for its biological implausibility (e.g., lack of local update rules for the parameters). Therefore, biologically plausible learning methods that rely on predictive coding (PC), a framework for describing information processing in the brain, are increasingly studied. Recent works prove that these methods can approximate BP up to a certain margin on multilayer perceptrons (MLPs), and asymptotically on any other complex model, and that zero-divergence inference learning (Z-IL), a variant of PC, is able to exactly implement BP on MLPs. However, the recent literature shows also that there is no biologically plausible method yet that can exactly replicate the weight update of BP on complex models. To fill this gap, in this paper, we generalize (PC and) Z-IL by directly defining them on computational graphs, and show that it can perform exact reverse differentiation. What results is the first biologically plausible algorithm that is equivalent to BP in the way of updating parameters on any neural network, providing a bridge between the interdisciplinary research of neuroscience and deep learning.

翻译：深度学习因人工神经网络的兴起而重新定义了人工智能领域，这些网络架构的灵感来源于大脑中的神经对应结构。多年来，人工智能与神经科学之间的这种二元性为这两个领域带来了巨大益处，使神经网络能够应用于数十种场景。这些网络使用了反向微分的一种高效实现——反向传播（BP）。然而，该算法常因其生物学上的不合理性（例如，缺乏参数的局部更新规则）而受到批评。因此，作为描述大脑中信息处理框架的预测编码（PC）所依赖的生物学合理学习方法正受到越来越多的研究。近期工作证明，这些方法能在多层感知机（MLP）上以一定误差逼近BP，并在任何其他复杂模型上渐进逼近；同时，PC的变体——零散度推理学习（Z-IL）能在MLP上精确实现BP。但近期文献也表明，目前尚无能精确复制复杂模型上BP权重更新的生物学合理方法。为填补这一空白，本文通过直接在计算图上定义PC和Z-IL来将其泛化，并证明其能执行精确的反向微分。由此产生了首个与BP在任意神经网络上参数更新方式等价的生物学合理算法，为神经科学和深度学习的跨学科研究架起了桥梁。