Deploying large language models on-device for always-on personal agents demands sustained inference from hardware tightly constrained in power, thermal envelope, and memory. We benchmark Qwen 2.5 1.5B (4-bit quantised) across four platforms: a Raspberry Pi 5 with Hailo-10H NPU, a Samsung Galaxy S24 Ultra, an iPhone 16 Pro, and a laptop NVIDIA RTX 4050 GPU. Using a fixed 258-token prompt over 20 warm-condition iterations per device, we measure throughput, latency, power, and thermal behaviour. For mobile platforms, thermal management supersedes peak compute as the primary constraint: the iPhone 16 Pro loses nearly half its throughput within two iterations, and the S24 Ultra suffers a hard OS-enforced GPU frequency floor that terminates inference entirely. On dedicated hardware, distinct constraints dominate: the RTX 4050 is bounded by its battery power ceiling, while the Hailo-10H is limited by on-module memory bandwidth. The RTX 4050 sustains 131.7 tok/s at 34.1 W; the Hailo-10H sustains 6.9 tok/s at under 2 W with near-zero variance, matching the RTX 4050 in energy proportionality at 19x lower throughput. Results should be interpreted as platform-level deployment characterisations for a single model and prompt type, reflecting hardware and software combined, rather than general claims about hardware capability alone.

翻译：在设备端部署大语言模型以实现始终在线的个人智能体，要求推理过程在功率、热设计和内存均受严格约束的硬件上持续运行。我们对Qwen 2.5 1.5B（4位量化）模型在四个平台进行了基准测试：搭载Hailo-10H NPU的树莓派5、三星Galaxy S24 Ultra、iPhone 16 Pro以及搭载NVIDIA RTX 4050 GPU的笔记本电脑。通过在每个设备上对固定258词元提示词进行20次热启动迭代测量，我们评估了吞吐量、延迟、功耗与热行为特征。对于移动平台，热管理取代峰值计算能力成为首要约束：iPhone 16 Pro在两次迭代后吞吐量下降近半，而S24 Ultra因操作系统强制执行的GPU频率下限导致推理完全终止。在专用硬件上，不同约束条件占据主导：RTX 4050受限于电池功率上限，Hailo-10H则受制于模组内存储器带宽。RTX 4050在34.1W功耗下保持131.7 tok/s的持续吞吐量；Hailo-10H在低于2W功耗下以近乎零方差维持6.9 tok/s，虽吞吐量仅为前者的1/19，但其能量效率比例与RTX 4050相当。研究结果应视为特定模型与提示词类型在平台层面的部署特性表征——反映硬件与软件的组合效应，而非单纯硬件能力的普适性论断。