There is a large literature on the similarities and differences between biological neural circuits and deep artificial neural networks (DNNs). However, modern training of DNNs relies on several engineering tricks such as data batching, normalization, adaptive optimizers, and precise weight initialization. Despite their critical role in training DNNs, these engineering tricks are often overlooked when drawing parallels between biological and artificial networks, potentially due to a lack of evidence for their direct biological implementation. In this study, we show that Oja's plasticity rule partly overcomes the need for some engineering tricks. Specifically, under difficult, but biologically realistic learning scenarios such as online learning, deep architectures, and sub-optimal weight initialization, Oja's rule can substantially improve the performance of pure backpropagation. Our results demonstrate that simple synaptic plasticity rules can overcome challenges to learning that are typically overcome using less biologically plausible approaches when training DNNs.

翻译：关于生物神经回路与深度人工神经网络（DNNs）之间的异同已有大量文献。然而，现代DNNs的训练依赖于多种工程技巧，例如数据批处理、归一化、自适应优化器以及精确的权重初始化。尽管这些工程技巧在DNNs训练中至关重要，但在比较生物与人工网络时却常被忽视，这或许是由于缺乏其直接生物实现的证据。本研究表明，Oja可塑性规则在一定程度上克服了对某些工程技巧的需求。具体而言，在困难但生物学上合理的学习场景下——如在线学习、深层架构以及次优权重初始化——Oja规则能显著提升纯反向传播算法的性能。我们的结果表明，简单的突触可塑性规则能够克服学习中的挑战，而这些挑战在训练DNNs时通常需借助生物学合理性较低的方法来解决。