A cornerstone of our understanding of both biological and artificial neural networks is that they store information in the strengths of synaptic connections among the neurons. However, in contrast to the well-established theory for quantifying information encoded by the firing activity of neural networks, there does not exist a framework for quantifying information stored in the network's connection distribution itself. Here, we develop a theoretical framework for synaptic information by using densely connected Hebbian networks performing autoassociative memory tasks and by modeling data patterns to be stored as log-normal distributions. Specifically, we derive analytical approximations for Shannon mutual information between the data and singletons, pairs, and arbitrary n-tuples of synaptic connections within the network. Our framework corroborates well-established insights regarding pattern storage capacity, supports the principle of distributed coding in neural firing activities, and formalizes the heterogeneity inherent in information encoding across synapses in a network. Notably, it discovers synergistic interactions among synapses, revealing that the information encoded jointly by all the synapses exceeds the 'sum of its parts'. Taken together, this study introduces a powerful, interpretable framework for quantitatively understanding information storage in the synapses of neural networks, one that illustrates the duality of synaptic connectivity and neural population activity in learning and memory.

翻译：理解生物和人工神经网络的一个基石在于，它们将信息存储在神经元之间的突触连接强度中。然而，与量化神经网络放电活动所编码信息的成熟理论相比，目前尚缺乏用于量化网络连接分布本身所存储信息的框架。在此，我们通过使用执行自联想记忆任务的密集连接Hebbian网络，并将待存储的数据模式建模为对数正态分布，为突触信息开发了一个理论框架。具体而言，我们推导出数据与网络内单个、成对及任意n元组突触连接之间的香农互信息的解析近似。我们的框架验证了关于模式存储容量的成熟见解，支持神经放电活动中的分布式编码原理，并形式化了网络中跨突触信息编码所固有的异质性。值得注意的是，它揭示了突触间的协同相互作用，表明所有突触联合编码的信息超过了其“各部分之和”。综上所述，本研究引入了一个强大且可解释的框架，用于定量理解神经网络的突触信息存储，该框架阐明了突触连接与神经群体活动在学习与记忆中的二元性。