Efficient AI inference on AMD's Versal AI Engine (AIE) is challenging due to tightly coupled VLIW execution, explicit datapaths, and local memory management. Prior work focused on first-generation AIE kernel optimizations, without tackling full neural network execution across the 2D array. In this work, we present AIE4ML, the first comprehensive framework for converting AI models automatically into optimized firmware targeting the AIE-ML generation devices, also with forward compatibility for the newer AIE-MLv2 architecture. At the single-kernel level, we attain performance close to the architectural peak. At the graph and system levels, we provide a structured parallelization method that can scale across the 2D AIE-ML fabric and exploit its dedicated memory tiles to stay entirely on-chip throughout the model execution. As a demonstration, we designed a generalized and highly efficient linear-layer implementation with intrinsic support for fused bias addition and ReLU activation. Also, as our framework necessitates the generation of multi-layer implementations, our approach systematically derives deterministic, compact, and topology-optimized placements tailored to the physical 2D grid of the device through a novel graph placement and search algorithm. Finally, the framework seamlessly accepts quantized models imported from high-level tools such as hls4ml or PyTorch while preserving bit-exactness. In layer scaling benchmarks, we achieve up to 98.6% efficiency relative to the single-kernel baseline, utilizing 296 of 304 AIE tiles (97.4%) of the device with entirely on-chip data movement. With evaluations across real-world model topologies, we demonstrate that AIE4ML delivers GPU-class throughput under microsecond latency constraints, making it a practical companion for ultra-low-latency environments such as trigger systems in particle physics experiments.

翻译：在AMD Versal AI Engine（AIE）上实现高效AI推理具有挑战性，原因在于其紧耦合的超长指令字执行、显式数据通路和本地内存管理。先前研究主要集中于第一代AIE内核优化，未能解决跨二维阵列的完整神经网络执行问题。本研究提出AIE4ML——首个将AI模型自动转换为针对AIE-ML系列设备的优化固件的综合性框架，同时具备对新一代AIE-MLv2架构的前向兼容性。在单内核层面，我们实现了接近架构峰值的性能。在图级和系统级，我们提供了一种结构化并行化方法，能够跨二维AIE-ML架构扩展，并利用其专用内存瓦片在整个模型执行期间完全保持片上数据流动。作为示例，我们设计了一个通用且高效的线性层实现方案，其内置支持融合偏置加法与ReLU激活功能。此外，由于本框架需要生成多层实现方案，我们通过创新的图布局与搜索算法，系统化地推导出针对设备物理二维网格定制的确定性、紧凑且拓扑优化的布局方案。最后，该框架可无缝接收从hls4ml或PyTorch等高层次工具导入的量化模型，同时保持比特级精确度。在层级扩展基准测试中，我们实现了相对于单内核基线高达98.6%的效率，在完全采用片上数据传输的情况下，利用了设备304个AIE瓦片中的296个（97.4%）。通过对实际模型拓扑的评估，我们证明AIE4ML能在微秒级延迟约束下提供GPU级吞吐量，使其成为粒子物理实验触发系统等超低延迟环境的实用解决方案。