Sequence modeling faces challenges in capturing long-range dependencies across diverse tasks. Recent linear and transformer-based forecasters have shown superior performance in time series forecasting. However, they are constrained by their inherent inability to effectively address long-range dependencies in time series data, primarily due to using fixed-size inputs for prediction. Furthermore, they typically sacrifice essential temporal correlation among consecutive training samples by shuffling them into mini-batches. To overcome these limitations, we introduce a fast and effective Spectral Attention mechanism, which preserves temporal correlations among samples and facilitates the handling of long-range information while maintaining the base model structure. Spectral Attention preserves long-period trends through a low-pass filter and facilitates gradient to flow between samples. Spectral Attention can be seamlessly integrated into most sequence models, allowing models with fixed-sized look-back windows to capture long-range dependencies over thousands of steps. Through extensive experiments on 11 real-world time series datasets using 7 recent forecasting models, we consistently demonstrate the efficacy of our Spectral Attention mechanism, achieving state-of-the-art results.

翻译：序列建模在捕捉跨不同任务的长程依赖方面面临挑战。近年来，基于线性模型和Transformer的预测器在时间序列预测中展现出卓越性能。然而，这些方法受限于其固有缺陷，无法有效处理时间序列数据中的长程依赖，主要原因在于它们使用固定尺寸的输入进行预测。此外，这些方法通常通过将连续训练样本随机打乱成小批量来牺牲样本间关键的时间相关性。为克服这些局限性，我们提出了一种快速有效的谱注意力机制，该机制在保持基础模型结构的同时，既保留了样本间的时间相关性，又促进了长程信息的处理。谱注意力通过低通滤波器保留长周期趋势，并促进梯度在样本间流动。该机制可无缝集成到大多数序列模型中，使具有固定尺寸回溯窗口的模型能够捕捉数千步范围内的长程依赖。通过在11个真实世界时间序列数据集上使用7种最新预测模型进行广泛实验，我们一致证明了谱注意力机制的有效性，并取得了最先进的预测结果。