Deep learning of artificial neural networks (ANNs) is creating highly functional processes that are, unfortunately, nearly as hard to interpret as their biological counterparts. Identification of functional modules in natural brains plays an important role in cognitive and neuroscience alike, and can be carried out using a wide range of technologies such as fMRI, EEG/ERP, MEG, or calcium imaging. However, we do not have such robust methods at our disposal when it comes to understanding functional modules in artificial neural networks. Ideally, understanding which parts of an artificial neural network perform what function might help us to address a number of vexing problems in ANN research, such as catastrophic forgetting and overfitting. Furthermore, revealing a network's modularity could improve our trust in them by making these black boxes more transparent. Here, we introduce a new information-theoretic concept that proves useful in understanding and analyzing a network's functional modularity: the relay information $I_R$. The relay information measures how much information groups of neurons that participate in a particular function (modules) relay from inputs to outputs. Combined with a greedy search algorithm, relay information can be used to identify computational modules in neural networks. We also show that the functionality of modules correlates with the amount of relay information they carry.

翻译：人工神经网络（ANNs）的深度学习正在生成高度功能化的处理过程，但遗憾的是，这些过程与其生物对应物一样难以解释。在自然大脑中识别功能模块在认知科学和神经科学中均发挥着重要作用，并可通过fMRI、EEG/ERP、MEG或钙成像等广泛技术实现。然而，在理解人工神经网络中的功能模块时，我们缺乏如此可靠的方法。理想情况下，了解人工神经网络的哪些部分执行何种功能，可能有助于解决ANN研究中一系列棘手问题，如灾难性遗忘和过拟合。此外，揭示网络的模块性可通过使这些黑箱模型更加透明来增强我们对它们的信任。在此，我们引入一个在理解和分析网络功能模块性方面证明有用的新信息论概念：中继信息 $I_R$。中继信息度量参与特定功能（模块）的神经元群从输入到输出中继的信息量。结合贪心搜索算法，中继信息可用于识别神经网络中的计算模块。我们还证明，模块的功能性与其携带的中继信息量呈相关性。