We present STanHop-Net (Sparse Tandem Hopfield Network) for multivariate time series prediction with memory-enhanced capabilities. At the heart of our approach is STanHop, a novel Hopfield-based neural network block, which sparsely learns and stores both temporal and cross-series representations in a data-dependent fashion. In essence, STanHop sequentially learn temporal representation and cross-series representation using two tandem sparse Hopfield layers. In addition, StanHop incorporates two additional external memory modules: a Plug-and-Play module and a Tune-and-Play module for train-less and task-aware memory-enhancements, respectively. They allow StanHop-Net to swiftly respond to certain sudden events. Methodologically, we construct the StanHop-Net by stacking STanHop blocks in a hierarchical fashion, enabling multi-resolution feature extraction with resolution-specific sparsity. Theoretically, we introduce a sparse extension of the modern Hopfield model (Generalized Sparse Modern Hopfield Model) and show that it endows a tighter memory retrieval error compared to the dense counterpart without sacrificing memory capacity. Empirically, we validate the efficacy of our framework on both synthetic and real-world settings.

翻译：本文提出STanHop-Net（稀疏串联Hopfield网络），用于具备记忆增强能力的多元时间序列预测。该框架的核心是STanHop——一种基于Hopfield的新型神经网络模块，能以数据依赖方式稀疏学习并存储时序与跨序列的表示。本质而言，STanHop通过两个串联的稀疏Hopfield层依次学习时序表征与跨序列表征。此外，STanHop整合了两种外部记忆模块：即插即用模块与调谐即用模块，分别实现无训练型与任务感知型记忆增强。这些模块使STanHop-Net能够快速响应特定突发事件。方法论层面，我们通过层级堆叠STanHop块构建STanHop-Net，实现具有分辨率特异性稀疏机制的多分辨率特征提取。理论层面，我们提出现代Hopfield模型的稀疏扩展（广义稀疏现代Hopfield模型），并证明其在不牺牲记忆容量的前提下，相较于稠密版本具有更紧密的记忆检索误差界。实证层面，我们在合成数据与真实场景中验证了该框架的有效性。