As custom hardware accelerators become increasingly central to machine learning workloads, efficient data transfer is critical for maximizing accelerator performance on linear algebra kernels. AXI4MLIR, an extension of the Multi-Level Intermediate Representation (MLIR) compiler framework for automated generation of host-accelerator driver code, incurs significant runtime overhead due to non-zero-copy CPU-accelerator data movement. During transfers from the host to the accelerator, data is copied from heap-allocated memory buffers into contiguous Direct Memory Access (DMA)-mapped buffers. This work identifies this copy as a redundant staging operation and eliminates it through zero-copy data movement. The optimization extends accel, an MLIR dialect introduced by AXI4MLIR, and implements lowering support that allocates buffers directly within DMA-mapped memory, thereby omitting the staging copy. We evaluate the proposed scheme using a configurable matrix-matrix multiplication accelerator and show that the zero-copy optimization reduces main memory data movement by up to 2x, increasing overall accelerator utilization.

翻译：随着定制硬件加速器在机器学习工作负载中日益核心，高效的数据传输对于最大化线性代数内核上的加速器性能至关重要。AXI4MLIR是多级中间表示（MLIR）编译器框架的扩展，用于自动生成主机-加速器驱动代码，但由于非零拷贝的CPU-加速器数据移动而引入显著的运行时开销。在从主机到加速器的传输过程中，数据从堆分配的缓冲区复制到连续的DMA映射缓冲区。本研究将此复制操作识别为冗余的分阶段操作，并通过零拷贝数据移动消除它。该优化扩展了AXI4MLIR引入的MLIR方言accel，并实现了降低支持，直接在DMA映射内存中分配缓冲区，从而省略了分阶段复制。我们使用可配置的矩阵-矩阵乘法加速器评估所提出的方案，并表明零拷贝优化将主内存数据移动减少多达2倍，提高了整体加速器利用率。