Major challenges in LLMs inference remain frequent memory bandwidth bottlenecks, computational redundancy, and inefficiencies in long-sequence processing. To address these issues, we propose LLM-CoOpt, a comprehensive algorithmhardware co-design framework aimed at improving both throughput and latency in LLM inference. LLM-CoOpt integrates three key strategies: (1) Key-Value Cache Optimization, termed Opt-KV, which improves memory access efficiency by optimizing both KV cache write and read paths, and introduces FP8 quantization to reduce memory footprint while maintaining accuracy; (2) Grouped-Query Attention for Computational Efficiency, termed Opt-GQA, which reduces the overall computational complexity by restructuring multi-head self-attention into grouped-query attention with shared key-value projections, enabling higher throughput and lower resource consumption; (3) Paged Attention for Long- Sequence Processing, termed Opt-Pa, which adopts a two-step strategy to first segment long sequences into manageable chunks and then apply lazy memory mapping and computation, significantly reducing memory pressure and improving performance on long-context inputs.Experiments on the LLaMa-13BGPTQ model demonstrate that LLM-CoOpt increases inference throughput by up to 13.43%, reduces latency by up to 16.79%, and maintains model accuracy. These results confirm that LLM-CoOpt provides a practical, high-performance optimization path for real-world inference of large-scale language models.

翻译：大语言模型推理面临的主要挑战包括频繁的内存带宽瓶颈、计算冗余以及长序列处理效率低下。为解决这些问题，我们提出LLM-CoOpt——一个旨在提升大语言模型推理吞吐量与延迟的算法-硬件协同设计综合框架。LLM-CoOpt整合了三大核心策略：(1) 键值缓存优化（Opt-KV），通过优化KV缓存的写入与读取路径提升内存访问效率，并引入FP8量化在保持精度的同时降低内存占用；(2) 面向计算效率的分组查询注意力（Opt-GQA），通过将多头自注意力重构为共享键值投影的分组查询注意力机制，降低整体计算复杂度，实现更高吞吐量与更低资源消耗；(3) 面向长序列处理的分页注意力（Opt-Pa），采用两步策略：先将长序列分割为可管理的块，再应用惰性内存映射与计算，显著降低长上下文输入的内存压力并提升性能。在LLaMa-13BGPTQ模型上的实验表明，LLM-CoOpt最高可提升推理吞吐量13.43%，降低延迟16.79%，且保持模型精度。这些结果证实LLM-CoOpt为实际场景中的大规模语言模型推理提供了实用高效的高性能优化路径。