Data movement overheads increase the inference latency of state-of-the-art large language models (LLMs). These models commonly use the bfloat16 (BF16) format for stable training. Floating-point standards allocate eight bits to the exponent, but our profiling reveals that exponent streams exhibit fewer than 3 bits Shannon entropy, indicating high inherent compressibility. To exploit this potential, we propose LEXI, a novel lossless exponent compression scheme based on Huffman coding. LEXI compresses activations and caches on the fly while storing compressed weights for just-in-time decompression near compute, without sacrificing system throughput and model accuracy. The codecs at the ingress and egress ports of network-on-chip routers sustain the maximum link bandwidth via multi-lane LUT decoders, incurring only 0.09 percent area and energy overheads with GF 22 nm technology. LEXI reduces inter-chiplet communication and end-to-end inference latencies by 33-45 percent and 30-35 percent on modern Jamba, Zamba, and Qwen LLMs implemented on a homogeneous chiplet architecture.

翻译：数据移动开销增加了最先进大语言模型（LLMs）的推理延迟。这些模型通常使用bfloat16（BF16）格式以保持训练稳定性。浮点数标准为指数分配了8个比特，但我们的性能分析表明指数流呈现少于3比特的香农熵，表明其具有固有的高可压缩性。为利用这一潜力，我们提出了LEXI——一种基于霍夫曼编码的新型无损指数压缩方案。LEXI在传输过程中动态压缩激活值和缓存，同时存储压缩后的权重以便在计算单元附近进行即时解压，且不牺牲系统吞吐量和模型精度。片上网络路由器入口与出口端口的编解码器通过多路查找表解码器维持最大链路带宽，在GF 22 nm工艺下仅产生0.09%的面积与能耗开销。在采用同构芯粒架构实现的现代Jamba、Zamba及Qwen大语言模型上，LEXI将芯粒间通信延迟和端到端推理延迟分别降低了33-45%和30-35%。