The transformer is a neural network component that can be used to learn useful representations of sequences or sets of datapoints. The transformer has driven recent advances in natural language processing, computer vision, and spatio-temporal modelling. There are many introductions to transformers, but most do not contain precise mathematical descriptions of the architecture and the intuitions behind the design choices are often also missing. Moreover, as research takes a winding path, the explanations for the components of the transformer can be idiosyncratic. In this note we aim for a mathematically precise, intuitive, and clean description of the transformer architecture.

翻译：Transformer是一种神经网络组件，可用于学习序列或数据集的有用表示。Transformer推动了自然语言处理、计算机视觉和时空建模等领域的最新进展。尽管已有许多关于Transformer的介绍，但大多数缺乏对架构的精确数学描述，且设计选择背后的直觉也常被忽略。此外，由于研究路径迂回曲折，对Transformer各组件的解释往往带有特殊性。本文旨在对Transformer架构提供数学精确、直观且清晰的描述。