Transformer models represent the cutting edge of Deep Neural Networks (DNNs) and excel in a wide range of machine learning tasks. However, processing these models demands significant computational resources and results in a substantial memory footprint. While In-memory Computing (IMC) offers promise for accelerating Matrix-Vector Multiplications (MVMs) with high computational parallelism and minimal data movement, employing it for implementing other crucial operators within DNNs remains a formidable task. This challenge is exacerbated by the extensive use of Softmax and data-dependent matrix multiplications within the attention mechanism. Furthermore, existing IMC designs encounter difficulties in fully harnessing the benefits of analog MVM acceleration due to the area and energy-intensive nature of Analog-to-Digital Converters (ADCs). To tackle these challenges, we introduce a novel Compute Analog Content Addressable Memory (Compute-ACAM) structure capable of performing various non-MVM operations within Transformers. Together with the crossbar structure, our proposed RACE-IT accelerator enables efficient execution of all operations within Transformer models in the analog domain. Given the flexibility of our proposed Compute-ACAMs to perform arbitrary operations, RACE-IT exhibits adaptability to diverse non-traditional and future DNN architectures without necessitating hardware modifications. Leveraging the capability of Compute-ACAMs to process analog input and produce digital output, we also replace ADCs, thereby reducing the overall area and energy costs. By evaluating various Transformer models against state-of-the-art GPUs and existing IMC accelerators, RACE-IT increases performance by 10.7x and 5.9x, and reduces energy by 1193x, and 3.9x, respectively

翻译：Transformer模型代表了深度神经网络（DNNs）的前沿技术，在广泛的机器学习任务中表现卓越。然而，处理这些模型需要大量的计算资源，并导致显著的内存占用。尽管内存内计算（IMC）有望以高计算并行性和最小数据移动来加速矩阵向量乘法（MVMs），但将其用于实现DNNs中的其他关键算子仍是一项艰巨任务。这一挑战因注意力机制中广泛使用的Softmax和依赖数据的矩阵乘法而进一步加剧。此外，由于模数转换器（ADCs）在面积和能耗上的高要求，现有IMC设计在充分利用模拟MVM加速优势方面也面临困难。为应对这些挑战，我们提出了一种新颖的计算模拟内容可寻址存储器（Compute-ACAM）结构，能够执行Transformer中的各种非MVM操作。结合交叉开关结构，我们提出的RACE-IT加速器能够在模拟域中高效执行Transformer模型的所有操作。鉴于所提出的Compute-ACAM在执行任意操作上的灵活性，RACE-IT能够适应各种非传统及未来的DNN架构，而无需修改硬件。利用Compute-ACAM处理模拟输入并生成数字输出的能力，我们替换了ADCs，从而降低了总体面积和能耗成本。通过将多种Transformer模型与最先进的GPU及现有IMC加速器进行比较评估，RACE-IT的性能分别提升了10.7倍和5.9倍，能耗分别降低了1193倍和3.9倍。