Digital computing-in-memory (DCIM) has emerged as a promising solution for large language model (LLM) acceleration by minimizing data transfers between external DRAM and on-chip accelerators while maintaining high precision for superior accuracy. However, existing CIM architectures often overlook weight update latency, which becomes critical as LLM weights are far larger than a single CIM macro capacity. To address this issue, this paper proposes a read-compute/write (RCW) architecture that effectively minimizes weight update latency, along with a nonlinear operator fusion that further mitigates dependencyinduced latency. The proposed RCW reduces decoding computing latency by 21.59% on the Llama2-7B model. In addition, the nonlinear operator fusion mechanism achieves a 69.17% latency reduction through efficient partial accumulation and group-based approximation. Furthermore, a weight-stationary and output column stationary (WS-OCS) dataflow is introduced to reduce both external DRAM access and internal CIM weight updates by 51.6% and 87.6% respectively during the prefill phase of 1024 tokens, leading to an overall 49.76% latency reduction. Fabricated using TSMC 22 nm CMOS technology and operating at 100 MHz, the proposed RCW-CIM achieves 3.28 TOPS and 42.3 TOPS/W, enabling 4.2 ms prefill latency and 26.87 decoded tokens per second for the INT4-weight Llama2 model with dual DDR5-6400 memory.

翻译：数字存内计算（DCIM）通过最小化外部DRAM与片上加速器之间的数据传输，同时保持高精度以获得优越的准确性，已成为大语言模型（LLM）加速的一种有前景的解决方案。然而，现有的CIM架构常常忽视权重更新延迟问题，而随着LLM权重大幅超过单个CIM宏容量，这一问题变得至关重要。为解决此问题，本文提出了一种读写计算（RCW）架构，可有效最小化权重更新延迟，并结合非线性算子融合进一步减少依赖导致的延迟。所提出的RCW在Llama2-7B模型上将解码计算延迟降低了21.59%。此外，通过高效的部分累积和基于分组的近似方法，非线性算子融合机制实现了69.17%的延迟减少。同时，引入了权重固定和输出列固定（WS-OCS）数据流，在1024个token的预填充阶段将外部DRAM访问和内部CIM权重更新分别减少51.6%和87.6%，从而整体降低49.76%的延迟。采用台积电22纳米CMOS工艺制造并在100MHz频率下运行时，所提出的RCW-CIM实现了3.28 TOPS和42.3 TOPS/W的能效，在配备双DDR5-6400内存的INT4权重的Llama2模型上实现了4.2毫秒的预填充延迟和每秒26.87个解码token。