We examine the assumption that the hidden-state vectors of recurrent neural networks (RNNs) tend to form clusters of semantically similar vectors, which we dub the clustering hypothesis. While this hypothesis has been assumed in the analysis of RNNs in recent years, its validity has not been studied thoroughly on modern neural network architectures. We examine the clustering hypothesis in the context of RNNs that were trained to recognize regular languages. This enables us to draw on perfect ground-truth automata in our evaluation, against which we can compare the RNN's accuracy and the distribution of the hidden-state vectors. We start with examining the (piecewise linear) separability of an RNN's hidden-state vectors into semantically different classes. We continue the analysis by computing clusters over the hidden-state vector space with multiple state-of-the-art unsupervised clustering approaches. We formally analyze the accuracy of computed clustering functions and the validity of the clustering hypothesis by determining whether clusters group semantically similar vectors to the same state in the ground-truth model. Our evaluation supports the validity of the clustering hypothesis in the majority of examined cases. We observed that the hidden-state vectors of well-trained RNNs are separable, and that the unsupervised clustering techniques succeed in finding clusters of similar state vectors.

翻译：我们研究了循环神经网络（RNN）隐藏状态向量倾向于形成语义相似向量簇的假设，并将其称为聚类假设。尽管近年来在分析RNN时已默认采用该假设，但其有效性在现代神经网络架构上尚未得到深入验证。我们在经过训练的、用于识别正则语言的RNN背景下检验聚类假设。这使我们能够利用完美的真实自动机模型进行评估，从而对比RNN的准确性与隐藏状态向量的分布。我们首先检验RNN隐藏状态向量在不同语义类别上的（分段线性）可分性，随后采用多种当前最先进的无监督聚类方法对隐藏状态向量空间进行聚类分析。通过判定聚类是否将语义相似的向量归于真实模型中的同一状态，我们正式分析了所得聚类函数的准确性以及聚类假设的有效性。实验评估表明，在大多数研究案例中聚类假设成立。我们观察到，经过良好训练的RNN的隐藏状态向量具有可分性，且无监督聚类技术能够成功找到相似状态向量的簇。