The deployment of Large Language Models (LLMs) on edge devices is fundamentally constrained by the "Memory Wall" the bottleneck where data movement latency outstrips arithmetic throughput. Standard inference runtimes often incur significant overhead through high-level abstractions, dynamic dispatch, and unaligned memory access patterns. In this work, we present a novel "Virtual Tensor Core" architecture implemented in software, optimized specifically for ARM64 microarchitectures (Apple Silicon). By bypassing standard library containers in favor of direct memory mapping (mmap) and implementing hand-tuned NEON SIMD kernels, we achieve a form of "Software-Defined Direct Memory Access (DMA)." Our proposed Tensor Virtualization Layout (TVL) guarantees 100% cache line utilization for weight matrices, while our zero-copy loader eliminates initialization latency. Experimental results on a 110M parameter model demonstrate a stable throughput of >60 tokens/second on M2 hardware. While proprietary hardware accelerators (e.g., Apple AMX) can achieve higher peak throughput, our architecture provides a fully open, portable, and deterministic reference implementation for studying the memory bottleneck on general-purpose ARM silicon, meeting the 200ms psycholinguistic latency threshold without opaque dependencies.

翻译：在边缘设备上部署大型语言模型（LLM）从根本上受到"内存墙"的限制——即数据移动延迟超过算术吞吐量的瓶颈。标准推理运行时通常因高层抽象、动态调度和非对齐内存访问模式而产生显著开销。本文提出一种新颖的软件实现"虚拟张量核心"架构，专门针对ARM64微架构（Apple Silicon）优化。通过绕过标准库容器，采用直接内存映射（mmap）并实现手动调优的NEON SIMD内核，我们实现了"软件定义直接内存访问（DMA）"。提出的张量虚拟化布局（TVL）保证权重矩阵实现100%缓存行利用率，而零拷贝加载器则消除了初始化延迟。在1.1亿参数模型上的实验结果表明，M2硬件上可实现>60词元/秒的稳定吞吐量。虽然专用硬件加速器（如Apple AMX）能实现更高的峰值吞吐量，但本架构为研究通用ARM芯片上的内存瓶颈提供了完全开源、可移植且确定性的参考实现，在无需依赖黑盒组件的情况下满足200毫秒心理语言学延迟阈值。