Time series analysis is vital for numerous applications, and transformers have become increasingly prominent in this domain. Leading methods customize the transformer architecture from NLP and CV, utilizing a patching technique to convert continuous signals into segments. Yet, time series data are uniquely challenging due to significant distribution shifts and intrinsic noise levels. To address these two challenges,we introduce the Sparse Vector Quantized FFN-Free Transformer (Sparse-VQ). Our methodology capitalizes on a sparse vector quantization technique coupled with Reverse Instance Normalization (RevIN) to reduce noise impact and capture sufficient statistics for forecasting, serving as an alternative to the Feed-Forward layer (FFN) in the transformer architecture. Our FFN-free approach trims the parameter count, enhancing computational efficiency and reducing overfitting. Through evaluations across ten benchmark datasets, including the newly introduced CAISO dataset, Sparse-VQ surpasses leading models with a 7.84% and 4.17% decrease in MAE for univariate and multivariate time series forecasting, respectively. Moreover, it can be seamlessly integrated with existing transformer-based models to elevate their performance.

翻译：时间序列分析对众多应用领域至关重要，Transformer模型在该领域中的应用日益突出。主流方法借鉴自然语言处理与计算机视觉领域的Transformer定制架构，采用分块技术将连续信号转换为片段。然而，时间序列数据因显著分布偏移和固有噪声水平而面临独特挑战。为应对这两大难题，我们提出稀疏向量量化无前馈网络Transformer（Sparse-VQ）。该方法利用稀疏向量量化技术结合反向实例归一化（RevIN），在降低噪声影响的同时捕获充分的统计量用于预测，作为Transformer架构中前馈层（FFN）的替代方案。这种无FFN的设计精简了参数量，提升了计算效率并降低了过拟合风险。通过在十个基准数据集（含新引入的CAISO数据集）上的评估，Sparse-VQ在单变量和多变量时间序列预测中分别实现7.84%和4.17%的平均绝对误差降低，超越主流模型。此外，该框架可无缝集成至现有基于Transformer的模型中，显著提升其性能。