Transformers are gaining increasing attention across different application domains due to their outstanding accuracy. However, these data-intensive models add significant performance demands to the existing computing architectures. Systolic arrays are spatial architectures that have been adopted by commercial AI computing platforms (like Google TPUs), due to their energy-efficient approach of data-reusability. However, these spatial architectures face a penalty in throughput and energy efficiency due to the need for input and output synchronization using First-In-First-Out (FIFO) buffers. This paper proposes a novel scalable systolic-array architecture featuring Diagonal-Input and Permutated weight-stationary (DiP) dataflow for the acceleration of matrix multiplication. The proposed architecture eliminates the synchronization FIFOs required by state-of-the-art weight stationary systolic arrays. Aside from the area, power, and energy savings achieved by eliminating these FIFOs, DiP architecture maximizes the computational resources (PEs) utilization. Thus, it outperforms the weight-stationary counterparts in terms of throughput by up to 50%. A comprehensive hardware design space exploration is demonstrated using commercial 22nm technology, highlighting the scalability advantages of DiP over the conventional approach across various dimensions where DiP offers improvement of energy efficiency per area up to 2.02x. Furthermore, DiP is evaluated using various transformer workloads from widely-used models, consistently outperforming TPU-like architectures, achieving energy improvements of up to 1.81x and latency improvements of up to 1.49x across a range of transformer workloads. At a 64x64 size with 4096 PEs, DiP achieves a peak performance of 8.2 TOPS with energy efficiency 9.55 TOPS/W.

翻译：Transformer模型因其卓越的准确性在不同应用领域受到越来越多的关注。然而，这些数据密集型模型对现有计算架构提出了显著的性能要求。脉动阵列作为一种空间架构，因其数据复用的高能效特性，已被商用AI计算平台（如谷歌TPU）所采用。然而，由于需要使用先进先出（FIFO）缓冲区进行输入输出同步，这些空间架构在吞吐量和能效方面面临损失。本文提出了一种新颖的可扩展脉动阵列架构，采用对角线输入与置换权重驻留（DiP）数据流以加速矩阵乘法运算。所提出的架构消除了最先进权重驻留脉动阵列所需的同步FIFO。除了通过消除这些FIFO实现面积、功耗和能耗的节省外，DiP架构还最大限度地提高了计算资源（处理单元）的利用率。因此，其吞吐量相比权重驻留方案最高可提升50%。通过采用商用22纳米工艺的全面硬件设计空间探索，展示了DiP相对于传统方法在多维度上的可扩展性优势，其中DiP的单位面积能效最高可提升2.02倍。此外，使用广泛采用模型中的多种Transformer工作负载对DiP进行评估，其性能持续优于类TPU架构，在各类Transformer工作负载中能效最高提升1.81倍，延迟最高降低1.49倍。在4096个处理单元的64×64规模下，DiP可实现8.2 TOPS的峰值性能，能效达9.55 TOPS/W。