Machine Learning (ML) operators are the building blocks to design ML models with various target applications. GEneral Matrix Multiplication (GEMM) operators are the backbone of ML models. They are notorious for being computationally expensive requiring billions of multiply-and-accumulate. Therefore, significant effort has been put to study and optimize the GEMM operators in order to speed up the execution of ML models. GPUs and accelerators are widely deployed to accelerate ML workloads by optimizing the execution of GEMM operators. Nonetheless, the performance of NonGEMM operators have not been studied as thoroughly as GEMMs. Therefore, this paper describes \bench, a benchmark to study NonGEMM operators. We first construct \bench using popular ML workloads from different domains, then perform case studies on various grade GPU platforms to analyze the behavior of NonGEMM operators in GPU accelerated systems. Finally, we present some key takeaways to bridge the gap between GEMM and NonGEMM operators and to offer the community with potential new optimization directions.

翻译：机器学习（ML）算子是构建面向各类目标应用的ML模型的基本单元。通用矩阵乘法（GEMM）运算是ML模型的支柱，因其需要数十亿级乘加运算而著称的高计算代价，业界已投入大量精力研究和优化GEMM算子以加速ML模型的执行。GPU和加速器通过优化GEMM算子的执行被广泛部署用于加速ML工作负载。然而，非GEMM算子的性能尚未像GEMM那样得到充分研究。为此，本文描述了一个用于研究非GEMM算子的基准测试\bench。我们首先利用来自不同领域的流行ML工作负载构建\bench，随后在多种级GPU平台上开展案例研究，分析非GEMM算子在GPU加速系统中的行为特征。最后，我们提出若干关键启示以弥合GEMM与非GEMM算子之间的性能差距，并为业界提供潜在的优化新方向。