We argue that the success of reservoir computing lies within the separation capacity of the reservoirs and show that the expected separation capacity of random linear reservoirs is fully characterised by the spectral decomposition of an associated generalised matrix of moments. Of particular interest are reservoirs with Gaussian matrices that are either symmetric or whose entries are all independent. In the symmetric case, we prove that the separation capacity always deteriorates with time; while for short inputs, separation with large reservoirs is best achieved when the entries of the matrix are scaled with a factor $\rho_T/\sqrt{N}$, where $N$ is the dimension of the reservoir and $\rho_T$ depends on the maximum length of the input time series. In the i.i.d. case, we establish that optimal separation with large reservoirs is consistently achieved when the entries of the reservoir matrix are scaled with the exact factor $1/\sqrt{N}$. We further give upper bounds on the quality of separation in function of the length of the time series. We complement this analysis with an investigation of the likelihood of this separation and the impact of the chosen architecture on separation consistency.
翻译:我们论证了储层计算的成功在于储层的分离能力,并表明随机线性储层的期望分离能力完全由关联的广义矩矩阵的谱分解刻画。特别值得关注的是高斯矩阵储层,其矩阵或为对称矩阵,或所有元素均相互独立。在对称情形下,我们证明分离能力总会随时间推移而恶化;而对于短序列输入,当矩阵元素按因子$\rho_T/\sqrt{N}$缩放时(其中$N$为储层维度,$\rho_T$取决于输入时间序列的最大长度),大型储层能实现最佳分离。在独立同分布情形下,我们证实当储层矩阵元素恰好按$1/\sqrt{N}$缩放时,大型储层能一致实现最优分离。进一步地,我们给出了分离质量随时间序列长度变化的上界。为完善该分析,我们还探讨了分离可能性的问题,并研究了所选架构对分离一致性的影响。