Recent years have witnessed the success of introducing Transformers to time series forecasting. From a data generation perspective, we illustrate that existing Transformers are susceptible to distribution shifts driven by temporal contexts, whether observed or unobserved. Such context-driven distribution shift (CDS) introduces biases in predictions within specific contexts and poses challenges for conventional training paradigm. In this paper, we introduce a universal calibration methodology for the detection and adaptation of CDS with a trained Transformer model. To this end, we propose a novel CDS detector, termed the "residual-based CDS detector" or "Reconditionor", which quantifies the model's vulnerability to CDS by evaluating the mutual information between prediction residuals and their corresponding contexts. A high Reconditionor score indicates a severe susceptibility, thereby necessitating model adaptation. In this circumstance, we put forth a straightforward yet potent adapter framework for model calibration, termed the "sample-level contextualized adapter" or "SOLID". This framework involves the curation of a contextually similar dataset to the provided test sample and the subsequent fine-tuning of the model's prediction layer with a limited number of steps. Our theoretical analysis demonstrates that this adaptation strategy is able to achieve an optimal equilibrium between bias and variance. Notably, our proposed Reconditionor and SOLID are model-agnostic and readily adaptable to a wide range of Transformers. Extensive experiments show that SOLID consistently enhances the performance of current SOTA Transformers on real-world datasets, especially on cases with substantial CDS detected by the proposed Reconditionor, thus validate the effectiveness of the calibration approach.

翻译：近年来，将Transformer引入时间序列预测取得了显著成功。从数据生成视角出发，我们证明现有Transformer易受时间上下文（无论是否可观测）驱动的分布偏移影响。这种上下文驱动的分布偏移（Context-Driven Distribution Shift, CDS）会在特定上下文中引入预测偏差，并对传统训练范式构成挑战。本文针对已训练的Transformer模型，提出一种用于检测与适应CDS的通用校准方法。为此，我们设计了新型CDS检测器——"基于残差的CDS检测器"（Reconditionor），通过评估预测残差与其对应上下文之间的互信息，量化模型对CDS的脆弱性。高Reconditionor得分表明模型脆弱性严重，需要进行模型适应。在此情况下，我们提出一个简单而有效的模型校准适配器框架——"样本级上下文适配器"（SOLID）。该框架通过构建与给定测试样本上下文相似的训练数据集，并对模型预测层进行有限步微调实现适应。理论分析证明，该适应策略能够实现偏差与方差的最优平衡。值得注意的是，所提出的Reconditionor与SOLID具有模型无关性，可便捷地应用于多种Transformer架构。大量实验表明，SOLID在真实数据集上持续提升当前最先进Transformer的性能，尤其在Reconditionor检测到显著CDS的场景中效果更为突出，从而验证了所提校准方法的有效性。