Sensory perception originates from the responses of sensory neurons, which react to a collection of sensory signals linked to various physical attributes of a singular perceptual object. Unraveling how the brain extracts perceptual information from these neuronal responses is a pivotal challenge in both computational neuroscience and machine learning. Here we introduce a statistical mechanical theory, where perceptual information is first encoded in the correlated variability of sensory neurons and then reformatted into the firing rates of downstream neurons. Applying this theory, we illustrate the encoding of motion direction using neural covariance and demonstrate high-fidelity direction recovery by spiking neural networks. Networks trained under this theory also show enhanced performance in classifying natural images, achieving higher accuracy and faster inference speed. Our results challenge the traditional view of neural covariance as a secondary factor in neural coding, highlighting its potential influence on brain function.

翻译：感官知觉源于感觉神经元的反应，这些神经元对与单一感知物体各种物理属性相关的感觉信号集合做出响应。揭示大脑如何从这些神经元反应中提取感知信息，是计算神经科学和机器学习领域的一项关键挑战。本文提出了一种统计力学理论，其中感知信息首先被编码在感觉神经元的相关变异性中，然后被重新格式化为下游神经元的放电频率。应用该理论，我们展示了利用神经协方差对运动方向进行编码，并通过脉冲神经网络实现了高保真度的方向恢复。在该理论指导下训练的神经网络在自然图像分类任务中也表现出增强的性能，实现了更高的准确率和更快的推理速度。我们的结果挑战了神经协方差作为神经编码中次要因素的传统观点，凸显了其对大脑功能的潜在影响。