We survey current developments in the approximation theory of sequence modelling in machine learning. Particular emphasis is placed on classifying existing results for various model architectures through the lens of classical approximation paradigms, and the insights one can gain from these results. We also outline some future research directions towards building a theory of sequence modelling.

翻译：本文综述了机器学习中序列建模近似理论的最新进展。重点在于通过经典近似范式的视角，对现有各类模型架构的结果进行分类，并阐释这些结果带来的洞察。我们还概述了未来构建序列建模理论的研究方向。