Time series prediction plays a crucial role in various industrial fields. In recent years, neural networks with a transformer backbone have achieved remarkable success in many domains, including computer vision and NLP. In time series analysis domain, some studies have suggested that even the simplest MLP networks outperform advanced transformer-based networks on time series forecast tasks. However, we believe these findings indicate there to be low-rank properties in time series sequences. In this paper, we consider the low-pass characteristics of transformers and try to incorporate the advantages of MLP. We adopt skip-layer connections inspired by Unet into traditional transformer backbone, thus preserving high-frequency context from input to output, namely U-shaped Transformer. We introduce patch merge and split operation to extract features with different scales and use larger datasets to fully make use of the transformer backbone. Our experiments demonstrate that the model performs at an advanced level across multiple datasets with relatively low cost.

翻译：时间序列预测在多个工业领域中发挥着关键作用。近年来，以Transformer为骨干的神经网络在计算机视觉和自然语言处理等诸多领域取得了显著成功。在时间序列分析领域中，一些研究表明，即便是最简单的MLP网络，在时间序列预测任务上也优于基于Transformer的先进网络。然而，我们认为这些发现表明时间序列序列存在低秩特性。本文考虑了Transformer的低通滤波特性，并尝试融合MLP的优势。我们将源自Unet的跨层连接引入传统Transformer骨干中，从而保留从输入到输出的高频上下文，即U型Transformer。我们引入补丁合并与拆分操作来提取多尺度特征，并使用更大的数据集以充分利用Transformer骨干。实验表明，该模型在多个数据集上以较低成本达到了先进水平。