The Transformer model has shown leading performance in time series forecasting. Nevertheless, in some complex scenarios, it tends to learn low-frequency features in the data and overlook high-frequency features, showing a frequency bias. This bias prevents the model from accurately capturing important high-frequency data features. In this paper, we undertook empirical analyses to understand this bias and discovered that frequency bias results from the model disproportionately focusing on frequency features with higher energy. Based on our analysis, we formulate this bias and propose Fredformer, a Transformer-based framework designed to mitigate frequency bias by learning features equally across different frequency bands. This approach prevents the model from overlooking lower amplitude features important for accurate forecasting. Extensive experiments show the effectiveness of our proposed approach, which can outperform other baselines in different real-world time-series datasets. Furthermore, we introduce a lightweight variant of the Fredformer with an attention matrix approximation, which achieves comparable performance but with much fewer parameters and lower computation costs. The code is available at: https://github.com/chenzRG/Fredformer

翻译：Transformer模型在时间序列预测中已展现出领先性能。然而，在一些复杂场景下，该模型倾向于学习数据中的低频特征而忽视高频特征，表现出频率偏差。这种偏差阻碍了模型准确捕捉重要的高频数据特征。本文通过实证分析来理解这种偏差，发现频率偏差源于模型对能量较高的频率特征给予了不成比例的关注。基于分析，我们形式化地描述了这种偏差，并提出Fredformer——一种基于Transformer的框架，旨在通过平等学习不同频带的特征来减轻频率偏差。该方法防止了模型忽视对准确预测至关重要的较低振幅特征。大量实验证明了我们提出的方法的有效性，该方法在多个真实世界时间序列数据集上均能超越其他基线模型。此外，我们引入了一种采用注意力矩阵近似的Fredformer轻量级变体，该变体在参数数量和计算成本大幅降低的同时，仍能获得可比的性能。代码公开于：https://github.com/chenzRG/Fredformer