With the rapid development of artificial intelligence, long short term memory (LSTM), one kind of recurrent neural network (RNN), has been widely applied in time series prediction. Like RNN, Transformer is designed to handle the sequential data. As Transformer achieved great success in Natural Language Processing (NLP), researchers got interested in Transformer's performance on time series prediction, and plenty of Transformer-based solutions on long time series forecasting have come out recently. However, when it comes to financial time series prediction, LSTM is still a dominant architecture. Therefore, the question this study wants to answer is: whether the Transformer-based model can be applied in financial time series prediction and beat LSTM. To answer this question, various LSTM-based and Transformer-based models are compared on multiple financial prediction tasks based on high-frequency limit order book data. A new LSTM-based model called DLSTM is built and new architecture for the Transformer-based model is designed to adapt for financial prediction. The experiment result reflects that the Transformer-based model only has the limited advantage in absolute price sequence prediction. The LSTM-based models show better and more robust performance on difference sequence prediction, such as price difference and price movement.

翻译：随着人工智能的快速发展，长短期记忆网络（LSTM）作为循环神经网络（RNN）的一种，已广泛应用于时间序列预测。与RNN相似，Transformer被设计用于处理序列数据。由于Transformer在自然语言处理（NLP）领域取得巨大成功，研究者开始关注其在时间序列预测上的表现，近期涌现了大量基于Transformer的长时间序列预测方案。然而，在金融时间序列预测领域，LSTM仍是主导架构。因此，本研究旨在回答：基于Transformer的模型能否应用于金融时间序列预测并超越LSTM？为回答此问题，我们在基于高频限价订单簿数据的多个金融预测任务上，对比了多种基于LSTM和基于Transformer的模型。我们构建了一种名为DLSTM的新LSTM模型，并设计了适配金融预测的Transformer新架构。实验结果表明，Transformer模型仅在绝对价格序列预测上具有有限优势，而LSTM模型在差分序列预测（如价格差与价格走势）上表现更优且更稳健。