We argue that the success of reservoir computing lies within the separation capacity of the reservoirs and show that the expected separation capacity of random linear reservoirs is fully characterised by the spectral decomposition of an associated generalised matrix of moments. Of particular interest are reservoirs with Gaussian matrices that are either symmetric or whose entries are all independent. In the symmetric case, we prove that the separation capacity always deteriorates with time; while for short inputs, separation with large reservoirs is best achieved when the entries of the matrix are scaled with a factor $\rho_T/\sqrt{N}$, where $N$ is the dimension of the reservoir and $\rho_T$ depends on the maximum length of the input time series. In the i.i.d. case, we establish that optimal separation with large reservoirs is consistently achieved when the entries of the reservoir matrix are scaled with the exact factor $1/\sqrt{N}$. We further give upper bounds on the quality of separation in function of the length of the time series. We complement this analysis with an investigation of the likelihood of this separation and the impact of the chosen architecture on separation consistency.
翻译:我们论证了储层计算成功的关键在于储层的分离容量,并表明随机线性储层的期望分离容量完全由关联广义矩矩阵的谱分解表征。特别值得关注的是高斯型矩阵储层,其矩阵要么对称,要么所有元素独立。在对称情形下,我们证明分离容量总会随时间退化;而对于短输入,当储层矩阵元素缩放因子取$\rho_T/\sqrt{N}$(其中$N$为储层维度,$\rho_T$取决于输入时间序列的最大长度)时,大型储层能获得最佳分离效果。在独立同分布情形下,我们确定当储层矩阵元素严格按因子$1/\sqrt{N}$缩放时,大型储层始终能实现最优分离。进一步地,我们给出了分离质量关于时间序列长度的上界。通过分析分离的似然性以及所选架构对分离一致性的影响,我们对上述分析进行了补充。