For a number of years since its introduction to hydrology, recurrent neural networks like long short-term memory (LSTM) have proven remarkably difficult to surpass in terms of daily hydrograph metrics on known, comparable benchmarks. Outside of hydrology, Transformers have now become the model of choice for sequential prediction tasks, making it a curious architecture to investigate. Here, we first show that a vanilla Transformer architecture is not competitive against LSTM on the widely benchmarked CAMELS dataset, and lagged especially for the high-flow metrics due to short-term processes. However, a recurrence-free variant of Transformer can obtain mixed comparisons with LSTM, producing the same Kling-Gupta efficiency coefficient (KGE), along with other metrics. The lack of advantages for the Transformer is linked to the Markovian nature of the hydrologic prediction problem. Similar to LSTM, the Transformer can also merge multiple forcing dataset to improve model performance. While the Transformer results are not higher than current state-of-the-art, we still learned some valuable lessons: (1) the vanilla Transformer architecture is not suitable for hydrologic modeling; (2) the proposed recurrence-free modification can improve Transformer performance so future work can continue to test more of such modifications; and (3) the prediction limits on the dataset should be close to the current state-of-the-art model. As a non-recurrent model, the Transformer may bear scale advantages for learning from bigger datasets and storing knowledge. This work serves as a reference point for future modifications of the model.

翻译：自循环神经网络（如长短期记忆网络，LSTM）引入水文学以来，在已知可比基准上，其在日径流过程指标方面长期难以被超越。然而在水文学之外，Transformer已成为序列预测任务的首选模型，这一架构值得深入探究。本研究首先证明，在广泛使用的CAMELS基准数据集上，标准Transformer架构未能超越LSTM，尤其在短期高流量过程指标上表现滞后。但一种无循环变体Transformer与LSTM的对比结果呈现混合特征，两者在克林-古普塔效率系数及其他指标上表现相当。Transformer缺乏优势的根本原因在于水文预测问题的马尔可夫特性。与LSTM类似，Transformer同样可通过融合多个强迫数据集提升模型性能。尽管Transformer的结果未超越当前最优模型，但我们获得了重要启示：（1）标准Transformer架构不适用于水文建模；（2）本文提出的无循环改进可提升Transformer性能，未来研究可继续探索此类改进方案；（3）当前数据集上的预测极限应接近现有最优模型水平。作为非循环模型，Transformer在处理更大数据集与知识存储方面具有规模优势。本研究可为该模型的未来改进提供参考基准。