Reservoir Computing (RC) models, a subclass of recurrent neural networks, are distinguished by their fixed, non-trainable input layer and dynamically coupled reservoir, with only the static readout layer being trained. This design circumvents the issues associated with backpropagating error signals through time, thereby enhancing both stability and training efficiency. RC models have been successfully applied across a broad range of application domains. Crucially, they have been demonstrated to be universal approximators of time-invariant dynamic filters with fading memory, under various settings of approximation norms and input driving sources. Simple Cycle Reservoirs (SCR) represent a specialized class of RC models with a highly constrained reservoir architecture, characterized by uniform ring connectivity and binary input-to-reservoir weights with an aperiodic sign pattern. For linear reservoirs, given the reservoir size, the reservoir construction has only one degree of freedom -- the reservoir cycle weight. Such architectures are particularly amenable to hardware implementations without significant performance degradation in many practical tasks. In this study we endow these observations with solid theoretical foundations by proving that SCRs operating in real domain are universal approximators of time-invariant dynamic filters with fading memory. Our results supplement recent research showing that SCRs in the complex domain can approximate, to arbitrary precision, any unrestricted linear reservoir with a non-linear readout. We furthermore introduce a novel method to drastically reduce the number of SCR units, making such highly constrained architectures natural candidates for low-complexity hardware implementations. Our findings are supported by empirical studies on real-world time series datasets.

翻译：储层计算（Reservoir Computing，RC）模型作为循环神经网络的一个子类，其显著特征在于具有固定的不可训练输入层和动态耦合的储层结构，仅静态读出层参与训练。这种设计规避了误差信号随时间反向传播的相关问题，从而同时提升了模型的稳定性和训练效率。RC模型已在众多应用领域取得成功应用。关键的是，研究已证明在不同近似范数和输入驱动源的设定下，该类模型能够作为具有衰减记忆特性的时不变动态滤波器的通用逼近器。简单循环储层（Simple Cycle Reservoirs，SCR）是一类具有高度约束储层架构的专用RC模型，其特征表现为均匀环形连接结构以及具有非周期符号模式的二元输入-储层权重。对于线性储层，在给定储层规模的前提下，其构建仅存在一个自由度——储层循环权重。此类架构特别适用于硬件实现，且在众多实际任务中不会导致显著的性能下降。本研究通过严格证明实域运行的SCR能够作为具有衰减记忆特性的时不变动态滤波器的通用逼近器，为这些观察结论奠定了坚实的理论基础。我们的研究成果补充了近期研究表明复域SCR能够以任意精度逼近具有非线性读出的无约束线性储层。此外，我们提出了一种全新方法以大幅减少SCR单元数量，使得此类高度约束的架构自然成为低复杂度硬件实现的理想候选方案。我们的研究结论得到了在真实世界时间序列数据集上的实证研究支持。