Modern GPUs increasingly rely on specialized and asynchronous hardware units to deliver high performance. Yet these units are often underutilized because today's GPU software stacks still organize programming and execution around a monolithic kernel model that mismatches asynchronous hardware. To address this issue, Virtual Decoupled Engines (VDCores) presents a new decoupled programming and execution model for asynchronous GPUs. VDCores abstracts asynchronous hardware execution units as resource isolated virtual cores and represents workloads as dependency-connected micro-operations (micro-ops). this abstraction removes static orchestration from the programmer, enables automatic overlap of memory and compute based on dependency and resource readiness, and thereby improves utilization of asynchronous hardware resources. Realizing such a decoupled abstraction efficiently on today's GPUs is itself challenging, VDCores addresses this through a GPU-specialized programming model and GPU runtime design that preserves the flexibility while minimizing implementation overhead. Across four LLM inference workloads on GH200, H100, and RTX 6000 Pro GPUs, VDCores significantly improves decoding throughput by 24% on average and by up to 77% under dynamic inputs, while reducing kernel programming and specialization effort by 90%. We have open sourced VDCores at https://github.com/vdcores/vdcores.

翻译：现代GPU日益依赖专用化和异步硬件单元以实现高性能。然而，由于当前GPU软件栈仍围绕与异步硬件不匹配的单一内核模型来组织编程与执行，这些单元往往利用率不足。为解决该问题，虚拟解耦引擎（VDCores）提出了一种适用于异步GPU的新型解耦编程与执行模型。VDCores将异步硬件执行单元抽象为资源隔离的虚拟核心，并将工作负载表示为依赖关联的微操作。该抽象消除了程序员的静态编排需求，能够基于依赖关系和资源就绪状态自动实现内存与计算的重叠，从而提升异步硬件资源的利用率。在当今GPU上高效实现此类解耦抽象本身具有挑战性。VDCores通过面向GPU特化的编程模型与运行时设计解决了该问题，在保持灵活性的同时最小化实现开销。在GH200、H100和RTX 6000 Pro GPU上针对四种大语言模型推理工作负载的测试表明，VDCores平均提升解码吞吐量24%，在动态输入下最高提升77%，同时将内核编程与特化工作量减少90%。我们已在https://github.com/vdcores/vdcores开源了VDCores。