The brain is not only constrained by energy needed to fuel computation, but it is also constrained by energy needed to form memories. Experiments have shown that learning simple conditioning tasks already carries a significant metabolic cost. Yet, learning a task like MNIST to 95% accuracy appears to require at least 10^{8} synaptic updates. Therefore the brain has likely evolved to be able to learn using as little energy as possible. We explored the energy required for learning in feedforward neural networks. Based on a parsimonious energy model, we propose two plasticity restricting algorithms that save energy: 1) only modify synapses with large updates, and 2) restrict plasticity to subsets of synapses that form a path through the network. Combining these two methods leads to substantial energy savings while only incurring a small increase in learning time. In biology networks are often much larger than the task requires. In particular in that case, large savings can be achieved. Thus competitively restricting plasticity helps to save metabolic energy associated to synaptic plasticity. The results might lead to a better understanding of biological plasticity and a better match between artificial and biological learning. Moreover, the algorithms might also benefit hardware because in electronics memory storage is energetically costly as well.

翻译：大脑不仅受限于支持计算所需的能量，还受限于形成记忆所需的能量。实验表明，学习简单的条件反射任务已会产生显著的代谢成本。然而，像MNIST分类任务达到95%的准确率似乎需要至少10^8次突触更新。因此，大脑很可能进化出了以尽可能少的能量进行学习的能力。我们探索了前馈神经网络中学习所需的能量。基于一个简约的能量模型，我们提出了两种节约能量的可塑性限制算法：1）仅修改具有较大更新量的突触；2）将可塑性限制在形成网络通路的突触子集上。结合这两种方法可在仅小幅增加学习时间的情况下实现大幅节能。在生物网络中，网络规模通常远大于任务需求——尤其在此情况下可实现显著节能。因此，竞争性限制可塑性有助于节省与突触可塑性相关的代谢能量。这些研究结果可能有助于更深入地理解生物可塑性，并促进人工学习与生物学习之间的更好匹配。此外，这些算法还可能对硬件领域有益，因为在电子设备中，记忆存储同样需要高昂的能量成本。