Transformers have demonstrated outstanding performance in many applications of deep learning. When applied to time series data, transformers require effective position encoding to capture the ordering of the time series data. The efficacy of position encoding in time series analysis is not well-studied and remains controversial, e.g., whether it is better to inject absolute position encoding or relative position encoding, or a combination of them. In order to clarify this, we first review existing absolute and relative position encoding methods when applied in time series classification. We then proposed a new absolute position encoding method dedicated to time series data called time Absolute Position Encoding (tAPE). Our new method incorporates the series length and input embedding dimension in absolute position encoding. Additionally, we propose computationally Efficient implementation of Relative Position Encoding (eRPE) to improve generalisability for time series. We then propose a novel multivariate time series classification (MTSC) model combining tAPE/eRPE and convolution-based input encoding named ConvTran to improve the position and data embedding of time series data. The proposed absolute and relative position encoding methods are simple and efficient. They can be easily integrated into transformer blocks and used for downstream tasks such as forecasting, extrinsic regression, and anomaly detection. Extensive experiments on 32 multivariate time-series datasets show that our model is significantly more accurate than state-of-the-art convolution and transformer-based models. Code and models are open-sourced at \url{https://github.com/Navidfoumani/ConvTran}.

翻译：Transformer在深度学习的诸多应用中展现了卓越性能。当应用于时间序列数据时，Transformer需要有效的位置编码来捕捉时间序列数据的顺序信息。位置编码在时间序列分析中的有效性尚未得到充分研究，并且仍存在争议，例如，是注入绝对位置编码还是相对位置编码，或是两者的结合更为优越。为澄清这一问题，我们首先回顾了现有绝对和相对位置编码方法在时间序列分类中的应用。随后，我们提出了一种专门针对时间序列数据的新型绝对位置编码方法，称为时间绝对位置编码（tAPE）。我们的新方法将序列长度和输入嵌入维度纳入绝对位置编码中。此外，我们还提出了计算高效的相对位置编码实现（eRPE），以提升时间序列的泛化能力。接着，我们提出了一种结合tAPE/eRPE与基于卷积的输入编码的新型多元时间序列分类（MTSC）模型，命名为ConvTran，以改善时间序列数据的位置与数据嵌入。所提出的绝对和相对位置编码方法简单高效，可轻松集成到Transformer模块中，并用于下游任务，如预测、外源回归和异常检测。在32个多元时间序列数据集上的广泛实验表明，我们的模型在准确性上显著优于基于卷积和Transformer的最先进模型。代码和模型已在 \url{https://github.com/Navidfoumani/ConvTran} 开源。