Chaotic dynamical systems (DS) are ubiquitous in nature and society. Often we are interested in reconstructing such systems from observed time series for prediction or mechanistic insight, where by reconstruction we mean learning geometrical and invariant temporal properties of the system in question (like attractors). However, training reconstruction algorithms like recurrent neural networks (RNNs) on such systems by gradient-descent based techniques faces severe challenges. This is mainly due to exploding gradients caused by the exponential divergence of trajectories in chaotic systems. Moreover, for (scientific) interpretability we wish to have as low dimensional reconstructions as possible, preferably in a model which is mathematically tractable. Here we report that a surprisingly simple modification of teacher forcing leads to provably strictly all-time bounded gradients in training on chaotic systems, and, when paired with a simple architectural rearrangement of a tractable RNN design, piecewise-linear RNNs (PLRNNs), allows for faithful reconstruction in spaces of at most the dimensionality of the observed system. We show on several DS that with these amendments we can reconstruct DS better than current SOTA algorithms, in much lower dimensions. Performance differences were particularly compelling on real world data with which most other methods severely struggled. This work thus led to a simple yet powerful DS reconstruction algorithm which is highly interpretable at the same time.

翻译：混沌动力系统（DS）在自然界和社会中普遍存在。我们通常希望从观测时间序列中重建此类系统，以实现预测或获取机理洞见——这里"重建"指学习系统（如吸引子）的几何结构及不变时间特性。然而，通过梯度下降技术训练循环神经网络（RNN）等重建算法时面临严峻挑战，这主要源于混沌系统中轨迹指数发散导致的梯度爆炸问题。此外，从科学可解释性角度出发，我们期望尽可能在低维空间中实现重建，且最好采用数学可处理的模型。本文发现，对教师强制方法进行一个简单改进，可以严格证明在混沌系统训练中梯度始终有界；当将此方法与可处理RNN设计（分段线性RNN，PLRNN）的简单架构重组相结合，即可在不超过观测系统维度的空间中实现精准重建。我们在多个动力系统上证明，经过这些改进后，我们能在更低维度上比当前最优算法更好地重建动力系统。在多数其他方法表现极差的真实世界数据上，性能差异尤为显著。本文最终提出了一种简单而强大的动力系统重建算法，同时具有高度可解释性。