Recent studies have demonstrated the great power of Transformer models for time series forecasting. One of the key elements that lead to the transformer's success is the channel-independent (CI) strategy to improve the training robustness. However, the ignorance of the correlation among different channels in CI would limit the model's forecasting capacity. In this work, we design a special Transformer, i.e., {\bf C}hannel {\bf A}ligned {\bf R}obust Blen{\bf d} Transformer (CARD for short), that addresses key shortcomings of CI type Transformer in time series forecasting. First, CARD introduces a channel-aligned attention structure that allows it to capture both temporal correlations among signals and dynamical dependence among multiple variables over time. Second, in order to efficiently utilize the multi-scale knowledge, we design a token blend module to generate tokens with different resolutions. Third, we introduce a robust loss function for time series forecasting to alleviate the potential overfitting issue. This new loss function weights the importance of forecasting over a finite horizon based on prediction uncertainties. Our evaluation of multiple long-term and short-term forecasting datasets demonstrates that CARD significantly outperforms state-of-the-art time series forecasting methods. The code is available at the following anonymous repository: \url{https://anonymous.4open.science/r/CARD-6EEC}
翻译:摘要:近期研究已证明Transformer模型在时间序列预测中的强大能力。其成功的关键要素之一是通道独立(CI)策略,该策略可提升训练鲁棒性。然而,CI策略忽略不同通道间关联性的缺陷会制约模型的预测能力。本文设计了一种专用Transformer——通道对齐鲁棒融合Transformer(简称CARD),旨在解决CI类Transformer在时间序列预测中的核心缺陷。首先,CARD引入通道对齐注意力结构,使其能够同时捕捉信号间的时间相关性以及多变量随时间变化的动态依赖关系。其次,为高效利用多尺度知识,我们设计了令牌融合模块以生成不同分辨率的令牌。第三,我们提出针对时间序列预测的鲁棒损失函数,用于缓解潜在的过拟合问题。该新型损失函数基于预测不确定性对有限预测窗口内的重要性进行加权。在多个长期与短期预测数据集上的评估结果表明,CARD显著优于当前最先进的时序预测方法。相关代码已公开于匿名仓库:\url{https://anonymous.4open.science/r/CARD-6EEC}