Large Language Models (LLMs) such as LLaMA and DeepSeek, are built on transformer architectures, which have become a standard model for achieving state-of-the-art performance in natural language processing tasks. Recently, there has been growing interest in deploying LLMs on edge devices. Although smaller LLM models are being proposed, they often still contain billions of parameters. Since edge devices are limited in their resources this poses a significant challenge for edge deployment. Compute-in-memory (CIM) is a promising architecture that addresses this by reducing data movement through the integration of computational logic directly into memory. However, existing CIM architectures support only static Multiply-Accumulate (MAC) operations which limit their configurability in supporting nonlinear operations and various types of transformer models. This paper presents a fully digital standard-cell SRAM-based CIM architecture accelerator for self-attention, called CIMple, designed to overcome these limitations, inside transformer models. The key contributions of CIMple are: 1) A novel dual-banked CIM-based fully digital self-attention accelerator using 8-bit parallel weight feeding. 2) A look-up-table (LUT) based fixed-point implementation reducing latency with minimal accuracy degradation. 3) A performance evaluation of a 32kb CIM-based self-attention accelerator implemented in 28nm, which achieves 26.1 TOPS/W at 0.85V and 2.31 TOPS/mm$^2$ at 1.2V, both with INT8 precision.

翻译：大型语言模型（LLMs），如LLaMA和DeepSeek，均基于Transformer架构构建，该架构已成为在自然语言处理任务中实现最先进性能的标准模型。近年来，将LLMs部署到边缘设备上的兴趣日益增长。尽管更小型的LLM模型正在被提出，但它们通常仍包含数十亿参数。由于边缘设备资源有限，这为其部署带来了重大挑战。存内计算（CIM）是一种有前景的架构，它通过将计算逻辑直接集成到存储器中来减少数据移动，从而解决这一问题。然而，现有的CIM架构仅支持静态乘累加（MAC）运算，限制了其在支持非线性运算及各类Transformer模型方面的可配置性。本文提出了一种名为CIMple的全数字标准单元SRAM基CIM架构加速器，专用于自注意力机制，旨在克服Transformer模型中的这些限制。CIMple的主要贡献包括：1）一种新颖的双存储体CIM全数字自注意力加速器，采用8位并行权重馈送；2）一种基于查找表（LUT）的定点实现，以最小精度损失降低延迟；3）在28nm工艺下实现的32kb CIM自注意力加速器性能评估显示，其在0.85V电压下达到26.1 TOPS/W，在1.2V电压下达到2.31 TOPS/mm²，两者均采用INT8精度。