The greatest demand for today's computing is machine learning. This paper analyzes three machine learning algorithms: transformers, spatial convolution, and FFT. The analysis is novel in three aspects. First, it measures the cost of memory access on an abstract memory hierarchy, instead of traditional time or space complexity. Second, the analysis is asymptotic and identifies the primary sources of the memory cost. Finally, the result is symbolic, which can be used to select algorithmic parameters such as the group size in grouped query attention for any dimension size and number of heads and the batch size for batched convolution for any image size and kernel size.

翻译：当今计算领域最大的需求是机器学习。本文分析了三种机器学习算法：Transformer、空间卷积和FFT。该分析在三个方面具有创新性。首先，它衡量了在抽象内存层次结构上的内存访问成本，而非传统的时间或空间复杂度。其次，该分析是渐近的，并确定了内存成本的主要来源。最后，结果是符号化的，可用于选择算法参数，例如分组查询注意力中的组大小（适用于任意维度大小和头数）以及批量卷积中的批次大小（适用于任意图像大小和卷积核大小）。