The transition toward localized intelligence through Small Language Models (SLMs) has intensified the need for rigorous performance characterization on resource-constrained edge hardware. However, objectively measuring the theoretical performance ceilings of diverse architectures across heterogeneous platforms remains a formidable challenge. In this work, we propose a systematic framework based on the Roofline model that unifies architectural primitives and hardware constraints through the lens of operational intensity (OI). By defining an inference-potential region, we introduce the Relative Inference Potential as a novel metric to compare efficiency differences between Large Language Models (LLMs) on the same hardware substrate. Extensive empirical analysis across diverse compute tiers reveals that variations in performance and OI are significantly influenced by sequence length. We further identify a critical regression in OI as model depth increases. Additionally, our findings highlight an efficiency trap induced by hardware heterogeneity and demonstrate how structural refinements, such as Multi-head Latent Attention (M LA), can effectively unlock latent inference potential across various hardware substrates. These insights provide actionable directions for hardware-software co-design to align neural structures with physical constraints in on-device intelligence. The released code is available in the Appendix C.

翻译：通过小型语言模型（SLMs）向本地化智能的转变，加大了对资源受限边缘硬件进行严格性能表征的需求。然而，客观地衡量异构平台上不同架构的理论性能上限仍然是一项艰巨的挑战。在本工作中，我们提出了一个基于屋顶线模型的系统性框架，该框架通过操作强度（OI）的视角统一了架构原语和硬件约束。通过定义推理潜力区域，我们引入了相对推理潜力这一新指标，用于比较相同硬件基底上大型语言模型（LLMs）之间的效率差异。跨不同计算层级的广泛实证分析表明，性能和OI的变化受序列长度影响显著。我们进一步发现，随着模型深度增加，OI存在一个关键的衰退现象。此外，我们的研究结果突显了由硬件异构性引发的效率陷阱，并展示了诸如多头潜在注意力（M LA）等结构优化如何能有效释放跨不同硬件基底的潜在推理能力。这些见解为硬件-软件协同设计提供了可行的方向，以使神经结构与设备端智能的物理约束相匹配。发布的代码可在附录C中获取。