Why do biological and artificial neurons sometimes modularise, each encoding a single meaningful variable, and sometimes entangle their representation of many variables? In this work, we develop a theory of when biologically inspired representations -- those that are nonnegative and energy efficient -- modularise with respect to source variables (sources). We derive necessary and sufficient conditions on a sample of sources that determine whether the neurons in an optimal biologically-inspired linear autoencoder modularise. Our theory applies to any dataset, extending far beyond the case of statistical independence studied in previous work. Rather, we show that sources modularise if their support is "sufficiently spread". From this theory, we extract and validate predictions in a variety of empirical studies on how data distribution affects modularisation in nonlinear feedforward and recurrent neural networks trained on supervised and unsupervised tasks. Furthermore, we apply these ideas to neuroscience data. First, we explain why two studies that recorded prefrontal activity in working memory tasks conflict on whether memories are encoded in orthogonal subspaces: the support of the sources differed due to a critical discrepancy in experimental protocol. Second, we use similar arguments to understand why preparatory and potent subspaces in RNN models of motor cortex are only sometimes orthogonal. Third, we study spatial and reward information mixing in entorhinal recordings, and show our theory matches data better than previous work. And fourth, we suggest a suite of surprising settings in which neurons can be (or appear) mixed selective, without requiring complex nonlinear readouts as in traditional theories. In sum, our theory prescribes precise conditions on when neural activities modularise, providing tools for inducing and elucidating modular representations in brains and machines.

翻译：为何生物神经元与人工神经元时而模块化——各自编码单一有意义的变量，时而将其对多变量的表征纠缠在一起？本研究针对生物启发表征（即非负且能量高效的表示）在何种条件下会相对于源变量（源）实现模块化，建立了理论框架。我们推导出源变量样本的充分必要条件，这些条件决定了最优生物启发式线性自编码器中神经元是否实现模块化。我们的理论适用于任意数据集，极大拓展了先前工作中仅研究统计独立性的局限。研究表明，当源变量的支撑集“充分分散”时，模块化现象才会出现。基于此理论，我们通过一系列实证研究提取并验证了数据分布如何影响非线性前馈与循环神经网络在监督与非监督任务中的模块化表现。此外，我们将这些观点应用于神经科学数据：首先，我们解释了为何两项关于工作记忆任务中前额叶活动记录的研究在记忆是否编码于正交子空间这一问题上存在分歧——实验协议的关键差异导致了源变量支撑集的不同；其次，运用类似论证阐明了运动皮层循环神经网络模型中预备子空间与潜能子空间仅有时正交的原因；第三，通过分析内嗅皮层记录中的空间信息与奖赏信息混合现象，证明我们的理论比先前研究更契合实验数据；最后，我们提出了一系列令人意外的情境，表明神经元可能（或看似）具有混合选择性，而无需传统理论中复杂的非线性读出机制。总之，我们的理论明确了神经活动实现模块化的精确条件，为在生物与机器系统中诱导和阐明模块化表征提供了方法论工具。