Large Language Models (LLMs) such as LLaMA and DeepSeek, are built on transformer architectures, which have become a standard model for achieving state-of-the-art performance in natural language processing tasks. Recently, there has been growing interest in deploying LLMs on edge devices. Although smaller LLM models are being proposed, they often still contain billions of parameters. Since edge devices are limited in their resources this poses a significant challenge for edge deployment. Compute-in-memory (CIM) is a promising architecture that addresses this by reducing data movement through the integration of computational logic directly into memory. However, existing CIM architectures support only static Multiply-Accumulate (MAC) operations which limit their configurability in supporting nonlinear operations and various types of transformer models. This paper presents a fully digital standard-cell SRAM-based CIM architecture accelerator for self-attention, called CIMple, designed to overcome these limitations, inside transformer models. The key contributions of CIMple are: 1) A novel dual-banked CIM-based fully digital self-attention accelerator using 8-bit parallel weight feeding. 2) A look-up-table (LUT) based fixed-point implementation reducing latency with minimal accuracy degradation. 3) A performance evaluation of a 32kb CIM-based self-attention accelerator implemented in 28nm, which achieves 26.1 TOPS/W at 0.85V and 2.31 TOPS/mm$^2$ at 1.2V, both with INT8 precision.

翻译：大型语言模型（如LLaMA和DeepSeek）基于Transformer架构构建，该架构已成为自然语言处理任务中实现最优性能的标准模型。近年来，将大型语言模型部署于边缘设备的兴趣日益增长。尽管小型化语言模型正在被提出，但其仍常包含数十亿参数。由于边缘设备资源受限，这对边缘部署构成了显著挑战。存内计算（CIM）通过将计算逻辑直接集成到存储器中减少数据移动，是一种有前景的架构。然而，现有CIM架构仅支持静态乘累加（MAC）操作，这限制了其在支持非线性操作及各类Transformer模型方面的可配置性。本文提出一种名为CIMple的全数字标准单元SRAM基CIM架构加速器，用于Transformer模型中的自注意力机制，旨在克服上述局限性。CIMple的核心贡献包括：1）新型双存储体CIM全数字自注意力加速器，采用8位并行权重馈入；2）基于查找表（LUT）的定点实现，以微小精度损失降低延迟；3）在28nm工艺下实现的32kb CIM自注意力加速器性能评估，在0.85V电压下达到26.1 TOPS/W，在1.2V电压下达到2.31 TOPS/mm²，均采用INT8精度。