Many state-of-the-art models trained on long-range sequences, for example S4, S5 or LRU, are made of sequential blocks combining State-Space Models (SSMs) with neural networks. In this paper we provide a PAC bound that holds for these kind of architectures with stable SSM blocks and does not depend on the length of the input sequence. Imposing stability of the SSM blocks is a standard practice in the literature, and it is known to help performance. Our results provide a theoretical justification for the use of stable SSM blocks as the proposed PAC bound decreases as the degree of stability of the SSM blocks increases.

翻译：许多在长序列上训练的最先进模型（例如S4、S5或LRU）由结合状态空间模型（SSMs）与神经网络的序列块构成。本文提出一种适用于具有稳定SSM块的此类架构的PAC界，该界不依赖于输入序列的长度。在文献中施加SSM块的稳定性是标准做法，且已知有助于提升性能。我们的结果为使用稳定SSM块提供了理论依据，因为所提出的PAC界随着SSM块稳定程度的增加而减小。