Chaotic systems are intrinsically sensitive to small errors, challenging efforts to construct predictive data-driven models of real-world dynamical systems such as fluid flows or neuronal activity. Prior efforts comprise either specialized models trained on individual time series, or foundation models trained on vast time series databases with little underlying dynamical structure. Motivated by dynamical systems theory, we present Panda, Patched Attention for Nonlinear DynAmics. We train Panda on a novel synthetic, extensible dataset of $2 \times 10^4$ chaotic dynamical systems that we discover using an evolutionary algorithm. Trained purely on simulated data, Panda exhibits emergent properties: zero-shot forecasting of unseen chaotic systems preserving both short-term accuracy and distributional measures, nonlinear resonance patterns in attention heads, and effective prediction of real-world experimental time series. Despite having been trained only on low-dimensional ordinary differential equations, Panda spontaneously develops the ability to predict partial differential equations without retraining. We also demonstrate a neural scaling law for differential equations, underscoring the potential of pre-trained models for probing abstract mathematical domains like nonlinear dynamics.

翻译：混沌系统对小误差具有内在敏感性，这对构建现实世界动力系统（如流体流动或神经元活动）的数据驱动预测模型提出了挑战。先前的研究主要包括针对单个时间序列训练的专业化模型，或在缺乏底层动力结构的大规模时间序列数据库上训练的基础模型。受动力系统理论启发，我们提出Panda（非线性动力学的分块注意力模型）。我们利用进化算法发现了一个包含$2 \times 10^4$个混沌动力系统的新型可扩展合成数据集，并在此基础上训练Panda。该模型仅使用模拟数据训练，却展现出涌现特性：对未见混沌系统进行零样本预测时，能同时保持短期精度和分布度量；注意力头中呈现非线性共振模式；并能有效预测真实世界实验时间序列。尽管仅基于低维常微分方程训练，Panda自发获得了预测偏微分方程的能力而无需重新训练。我们还证明了微分方程的神经缩放定律，这凸显了预训练模型在探索非线性动力学等抽象数学领域的潜力。