Matrix extensions have emerged as an essential feature in modern CPUs to address the surging demands of AI workloads. However, existing designs often incur substantial hardware and software design overhead. Tight coupling with the CPU pipeline complicates integration across diverse CPUs, while fine-grained synchronous instructions hinder the development of high-performance kernels. This paper proposes a unified and configurable CPU matrix extension architecture. By decoupling matrix units from the CPU pipeline, the design enables low-overhead integration while maintaining close coordination with existing compute and memory resources. The configurable matrix unit supports mixed-precision operations and adapts to diverse compute demands and memory bandwidth constraints. An asynchronous matrix multiplication abstraction with flexible granularity conceals hardware details, simplifies matrix-vector overlap, and supports a unified software stack. The architecture is integrated into four open-source CPU RTL platforms and evaluated on representative AI models. Matrix unit utilization under GEMM workloads exceeds 90% across all platforms. When configured with compute throughput and memory bandwidth comparable to Intel AMX, our design achieves speedups of 1.57x, 1.57x, and 2.31x on ResNet, BERT, and Llama3, with over 30% of the gains attributed to overlapped matrix-vector execution. A 4 TOPS@2GHz matrix unit occupies only 0.53 mm\textsuperscript{2} in 14nm CMOS. These results demonstrate strong cross-platform adaptability and effective hardware-software co-optimization, offering a practical matrix extension for the open-source community.

翻译：矩阵扩展已成为现代CPU中应对AI工作负载激增需求的关键特性。然而，现有设计往往带来显著的硬件与软件设计开销。与CPU流水线的紧耦合方式增加了跨不同CPU架构集成的复杂性，而细粒度同步指令则阻碍了高性能计算核心的开发。本文提出了一种统一且可配置的CPU矩阵扩展架构。通过将矩阵单元与CPU流水线解耦，该设计在保持与现有计算及存储资源紧密协同的同时，实现了低开销集成。可配置矩阵单元支持混合精度运算，可适应多样化的计算需求与内存带宽约束。一种具有灵活粒度的异步矩阵乘法抽象机制隐藏了硬件细节，简化了矩阵-向量重叠执行，并支持统一的软件栈。该架构已在四个开源CPU RTL平台上完成集成，并在代表性AI模型上进行了评估。在GEMM工作负载下，所有平台的矩阵单元利用率均超过90%。当配置为与Intel AMX相当的计算吞吐量与内存带宽时，我们的设计在ResNet、BERT和Llama3上分别实现了1.57倍、1.57倍和2.31倍的加速比，其中超过30%的性能提升归因于矩阵-向量重叠执行。在14nm CMOS工艺下，一个4 TOPS@2GHz的矩阵单元仅占用0.53 mm\textsuperscript{2}面积。这些结果证明了该架构强大的跨平台适应能力与有效的软硬件协同优化效果，为开源社区提供了一种实用的矩阵扩展方案。