Recurrent neural networks are a powerful means to cope with time series. We show how autoregressive linear, i.e., linearly activated recurrent neural networks (LRNNs) can approximate any time-dependent function f(t) given by a number of function values. The approximation can effectively be learned by simply solving a linear equation system; no backpropagation or similar methods are needed. Furthermore, and this is probably the main contribution of this article, the size of an LRNN can be reduced significantly in one step after inspecting the spectrum of the network transition matrix, i.e., its eigenvalues, by taking only the most relevant components. Therefore, in contrast to other approaches, we do not only learn network weights but also the network architecture. LRNNs have interesting properties: They end up in ellipse trajectories in the long run and allow the prediction of further values and compact representations of functions. We demonstrate this by several experiments, among them multiple superimposed oscillators (MSO), robotic soccer, and predicting stock prices. LRNNs outperform the previous state-of-the-art for the MSO task with a minimal number of units.

翻译：循环神经网络是处理时间序列的强大工具。我们展示了自回归线性（即线性激活的循环神经网络，LRNNs）如何能够近似任意由若干函数值给定的时间依赖函数f(t)。通过简单求解线性方程组即可有效学习该近似，无需反向传播或类似方法。此外，这可能是本文的主要贡献，在检查网络转移矩阵的谱（即其特征值）后，通过仅保留最相关的分量，LRNN的规模可在一步内大幅缩减。因此，与其他方法不同，我们不仅学习网络权重，还学习网络架构。LRNN具有有趣的性质：长期运行后它们会终止于椭圆轨迹，并允许预测后续值及函数的紧凑表示。我们通过多项实验证明了这一点，其中包括多重叠加振荡器（MSO）、机器人足球以及股票价格预测。LRNN以极少数量的单元在MSO任务上超越了先前的最优水平。