Chaotic dynamical systems (DS) are ubiquitous in nature and society. Often we are interested in reconstructing such systems from observed time series for prediction or mechanistic insight, where by reconstruction we mean learning geometrical and invariant temporal properties of the system in question (like attractors). However, training reconstruction algorithms like recurrent neural networks (RNNs) on such systems by gradient-descent based techniques faces severe challenges. This is mainly due to exploding gradients caused by the exponential divergence of trajectories in chaotic systems. Moreover, for (scientific) interpretability we wish to have as low dimensional reconstructions as possible, preferably in a model which is mathematically tractable. Here we report that a surprisingly simple modification of teacher forcing leads to provably strictly all-time bounded gradients in training on chaotic systems, and, when paired with a simple architectural rearrangement of a tractable RNN design, piecewise-linear RNNs (PLRNNs), allows for faithful reconstruction in spaces of at most the dimensionality of the observed system. We show on several DS that with these amendments we can reconstruct DS better than current SOTA algorithms, in much lower dimensions. Performance differences were particularly compelling on real world data with which most other methods severely struggled. This work thus led to a simple yet powerful DS reconstruction algorithm which is highly interpretable at the same time.

翻译：混沌动力系统在自然界和社会中普遍存在。我们通常希望从观测时间序列中重构此类系统，以实现预测或理解其机制——这里所说的重构，是指学习目标系统的几何与时间不变特性（如吸引子）。然而，通过基于梯度下降的技术训练循环神经网络等重构算法时，这类系统会面临严峻挑战。这主要源于混沌系统中轨迹的指数发散导致梯度爆炸。此外，为了（科学）可解释性，我们期望尽可能低维度的重构，最好采用数学上易于处理的模型。本文报告了一个令人惊讶的发现：对教师强制进行简单修改，即可在混沌系统训练中实现理论上严格有界的梯度，且与一种易于处理的RNN架构——分段线性RNN的简单结构调整相结合，能够在不超过观测系统维度的空间中实现忠实重构。我们在多个动力系统上证明，通过这些改进，我们能在远低于现有最先进算法所需的维度下更好地重构系统。在大多数其他方法难以处理的真实世界数据上，性能差异尤为显著。因此，本研究提出了一种简单而强大的动力系统重构算法，同时具备高度可解释性。