In neural networks, task-relevant information is represented jointly by groups of neurons. However, the specific way in which this mutual information about the classification label is distributed among the individual neurons is not well understood: While parts of it may only be obtainable from specific single neurons, other parts are carried redundantly or synergistically by multiple neurons. We show how Partial Information Decomposition (PID), a recent extension of information theory, can disentangle these different contributions. From this, we introduce the measure of "Representational Complexity", which quantifies the difficulty of accessing information spread across multiple neurons. We show how this complexity is directly computable for smaller layers. For larger layers, we propose subsampling and coarse-graining procedures and prove corresponding bounds on the latter. Empirically, for quantized deep neural networks solving the MNIST and CIFAR10 tasks, we observe that representational complexity decreases both through successive hidden layers and over training, and compare the results to related measures. Overall, we propose representational complexity as a principled and interpretable summary statistic for analyzing the structure and evolution of neural representations and complex systems in general.

翻译：在神经网络中，任务相关信息由神经元群体共同表征。然而，关于分类标签的互信息在单个神经元之间具体如何分布尚不明确：部分信息可能仅能从特定的单个神经元获取，而其他部分则由多个神经元冗余或协同地携带。我们展示了信息论最新扩展——部分信息分解（PID）如何厘清这些不同贡献。由此我们引入"表征复杂度"这一度量，它量化了跨越多个神经元的信息访问难度。我们证明了该复杂度对于较小层级可直接计算；对于较大层级，我们提出了子采样和粗粒化方法，并为后者证明了相应的界限。实验表明，在解决MNIST和CIFAR10任务的量化深度神经网络中，表征复杂度随隐藏层逐层深化及训练进程而降低，并将结果与相关度量进行比较。总体而言，我们提出表征复杂度作为分析神经表征结构与演化及一般复杂系统的原则性、可解释的综合统计量。