Extreme-edge scientific applications use machine learning models to analyze sensor data and make real-time decisions. Their stringent latency and throughput requirements demand small batch sizes and require that model weights remain fully on-chip. Spatial dataflow implementations are common for extreme-edge applications. Spatial dataflow works well for small networks, but it fails to scale to larger models due to inherent resource scaling limitations. AI Engines on modern FPGA SoCs offer a promising alternative with high compute density and additional on-chip memory. However, the architecture, programming model, and performance-scaling behavior of AI Engines differ fundamentally from those of the programmable logic, making direct comparison non-trivial and the benefits of using AI Engines unclear. This work addresses how and when extreme-edge scientific neural networks should be implemented on AI Engines versus programmable logic. We provide systematic architectural characterization and micro-benchmarking and introduce a latency-adjusted resource equivalence (LARE) metric that identifies when AI Engine implementations outperform programmable logic designs. We further propose spatial and API-level dataflow optimizations tailored to low-latency scientific inference. Finally, we demonstrate the successful deployment of end-to-end neural networks on AI Engines that cannot fit on programmable logic when using the hlsml toolchain.

翻译：极端边缘科学应用利用机器学习模型分析传感器数据并做出实时决策。其对延迟和吞吐量的严格约束要求采用小批量处理，并确保模型权重完全保持在片内。空间数据流实现是极端边缘应用的常见方案。空间数据流在小规模网络中表现良好，但由于资源缩放的内在限制，无法扩展至更大规模的模型。现代FPGA SoC上的AI引擎凭借高计算密度和额外片内存储器，提供了一种有前景的替代方案。然而，AI引擎的架构、编程模型和性能扩展行为与可编程逻辑存在根本性差异，这使得直接比较颇具挑战，且使用AI引擎的优势尚不明确。本研究探讨了在极端边缘科学神经网络中，何时以及如何在AI引擎与可编程逻辑之间进行选择实现。我们提供了系统的架构特征分析与微基准测试，并引入了延迟调整资源等价（LARE）指标，用于识别AI引擎实现优于可编程逻辑设计的情形。进一步提出了针对低延迟科学推理定制的空间和API级数据流优化。最后，我们成功演示了在AI引擎上部署完整端到端神经网络的案例——当使用hlsml工具链时，这些网络因资源限制无法适配至可编程逻辑。