Recurrent neural networks are a powerful means to cope with time series. We show how autoregressive linear, i.e., linearly activated recurrent neural networks (LRNNs) can approximate any time-dependent function f(t) given by a number of function values. The approximation can effectively be learned by simply solving a linear equation system; no backpropagation or similar methods are needed. Furthermore, and this is probably the main contribution of this article, the size of an LRNN can be reduced significantly in one step after inspecting the spectrum of the network transition matrix, i.e., its eigenvalues, by taking only the most relevant components. Therefore, in contrast to other approaches, we do not only learn network weights but also the network architecture. LRNNs have interesting properties: They end up in ellipse trajectories in the long run and allow the prediction of further values and compact representations of functions. We demonstrate this by several experiments, among them multiple superimposed oscillators (MSO), robotic soccer, and predicting stock prices. LRNNs outperform the previous state-of-the-art for the MSO task with a minimal number of units.

翻译：循环神经网络是处理时间序列的强大工具。我们展示了自回归线性（即线性激活的循环神经网络，LRNN）如何能够近似由若干函数值给出的任意时间相关函数 f(t)。这种近似可以通过简单地求解线性方程组来有效学习，无需反向传播或类似方法。更重要的是，这可能是本文的主要贡献：在检查网络转移矩阵的谱（即其特征值）后，通过仅保留最相关的成分，LRNN的规模可以在一瞬间显著缩减。因此，与其他方法不同，我们不仅学习网络权重，还学习网络架构。LRNN具有有趣的性质：长期会收敛到椭圆轨迹，并允许预测进一步的值以及函数的紧凑表示。我们通过多项实验证明了这一点，其中包括多重叠加振荡器（MSO）、机器人足球和股票价格预测。在使用最少单元数的MSO任务中，LRNN超越了之前的最先进水平。