We introduce a new approach to understanding trained sequence neural models: the Koopman Analysis of Neural Networks (KANN) method. Motivated by the relation between time-series models and self-maps, we compute approximate Koopman operators that encode well the latent dynamics. Unlike other existing methods whose applicability is limited, our framework is global, and it has only weak constraints over the inputs. Moreover, the Koopman operator is linear, and it is related to a rich mathematical theory. Thus, we can use tools and insights from linear analysis and Koopman Theory in our study. For instance, we show that the operator eigendecomposition is instrumental in exploring the dominant features of the network. Our results extend across tasks and architectures as we demonstrate for the copy problem, and ECG classification and sentiment analysis tasks.

翻译：我们引入了一种新的方法来理解经过训练的序列神经模型:神经网络库普曼分析(KANN)方法。根据时间序列模型和自我映射之间的关系,我们计算出潜伏动态的大致库普曼操作者。与其他现有方法不同,我们的框架是全球性的,对投入的限制也很小。此外,库普曼操作者是线性的,与丰富的数学理论有关。因此,我们可以在我们的研究中使用线性分析的工具和洞察力以及库普曼理论。例如,我们表明操作者eigendecomposition有助于探索网络的主导特征。我们的结果跨越了我们为复制问题展示的任务和结构,以及ECG分类和情感分析任务。