Optimal deployment of deep neural networks (DNNs) on state-of-the-art Systems-on-Chips (SoCs) is crucial for tiny machine learning (TinyML) at the edge. The complexity of these SoCs makes deployment non-trivial, as they typically contain multiple heterogeneous compute cores with limited, programmer-managed memory to optimize latency and energy efficiency. We propose HTVM - a compiler that merges TVM with DORY to maximize the utilization of heterogeneous accelerators and minimize data movements. HTVM allows deploying the MLPerf(TM) Tiny suite on DIANA, an SoC with a RISC-V CPU, and digital and analog compute-in-memory AI accelerators, at 120x improved performance over plain TVM deployment.

翻译：深度神经网络在先进系统级芯片上的最优部署对于边缘端微型机器学习至关重要。此类系统级芯片的复杂性使得部署具有挑战性——它们通常包含多个异构计算核心，且配备由程序员管理的有限内存，以优化延迟与能效。我们提出HTVM——一种融合TVM与DORY的编译器，旨在最大化异构加速器的利用率并最小化数据搬运。HTVM可将MLPerf® Tiny基准套件部署于DIANA（一款集成RISC-V CPU、数字与模拟存算一体AI加速器的系统级芯片），其性能较原始TVM部署提升120倍。