Despite considerable theoretical progress in the training of neural networks viewed as a multi-agent system of neurons, particularly concerning biological plausibility and decentralized training, their applicability to real-world problems remains limited due to scalability issues. In contrast, error-backpropagation has demonstrated its effectiveness for training deep networks in practice. In this study, we propose a local objective for neurons that, when pursued by neurons individually, align them to exhibit similarities to error-backpropagation in terms of efficiency and scalability during training. For this purpose, we examine a neural network comprising decentralized, self-interested neurons seeking to maximize their local objective -- attention from subsequent layer neurons -- and identify the optimal strategy for neurons. We also analyze the relationship between this strategy and backpropagation, establishing conditions under which the derived strategy is equivalent to error-backpropagation. Lastly, we demonstrate the learning capacity of these multi-agent neural networks through experiments on three datasets and showcase their superior performance relative to error-backpropagation in a catastrophic forgetting benchmark.

翻译：尽管将神经网络视为神经元的多智能体系统在理论研究上取得了显著进展（尤其是在生物合理性和分散训练方面），但由于可扩展性问题，其在实际问题中的适用性仍然有限。相比之下，误差反向传播在实践中已证明了其在训练深度网络方面的有效性。在本研究中，我们提出了一种局部目标函数，当神经元各自追求该目标时，它们能在训练过程中表现出与误差反向传播相似的效率和可扩展性。为此，我们考察了一个由分散、自利神经元组成的神经网络——这些神经元试图最大化其局部目标（即来自后续层神经元的注意力），并确定了神经元的最优策略。我们还分析了该策略与反向传播之间的关系，建立了所推导策略与误差反向传播等价的条件。最后，我们通过三个数据集上的实验展示了这些多智能体神经网络的学习能力，并在灾难性遗忘基准测试中证明了其相对于误差反向传播的优越性能。