Deep Neural Networks (DNNs) have achieved remarkable success across various intelligent tasks but encounter performance and energy challenges in inference execution due to data movement bottlenecks. We introduce DataMaestro, a versatile and efficient data streaming unit that brings the decoupled access/execute architecture to DNN dataflow accelerators to address this issue. DataMaestro supports flexible and programmable access patterns to accommodate diverse workload types and dataflows, incorporates fine-grained prefetch and addressing mode switching to mitigate bank conflicts, and enables customizable on-the-fly data manipulation to reduce memory footprints and access counts. We integrate five DataMaestros with a Tensor Core-like GeMM accelerator and a Quantization accelerator into a RISC-V host system for evaluation. The FPGA prototype and VLSI synthesis results demonstrate that DataMaestro helps the GeMM core achieve nearly 100% utilization, which is 1.05-21.39x better than state-of-the-art solutions, while minimizing area and energy consumption to merely 6.43% and 15.06% of the total system.

翻译：深度神经网络（DNN）在各种智能任务中取得了显著成功，但由于数据移动瓶颈，在推理执行中面临性能和能耗挑战。为此，我们提出了DataMaestro，一种通用高效的数据流单元，它将解耦访问/执行架构引入DNN数据流加速器以解决此问题。DataMaestro支持灵活可编程的访问模式，以适应多样化的工作负载类型和数据流；它集成了细粒度预取和寻址模式切换机制，以缓解存储体冲突；并支持可定制的实时数据操作，以减少内存占用和访问次数。我们将五个DataMaestro单元与一个类Tensor Core的GeMM加速器及一个量化加速器集成到RISC-V主机系统中进行评估。FPGA原型和VLSI综合结果表明，DataMaestro帮助GeMM核心实现了近100%的利用率，比现有先进方案提升了1.05至21.39倍，同时将面积和能耗分别降至系统总开销的仅6.43%和15.06%。