The emergence of large-scale, sparse, multimodal, and agentic AI models has coincided with a shift in hardware toward supernode architectures that integrate hundreds to thousands of accelerators with ultra-low-latency interconnects and unified memory pools. However, existing AI frameworks are not designed to exploit these architectures efficiently, leading to high programming complexity, load imbalance, and poor memory utilization. In this paper, we propose a supernode-affinity AI framework that treats the supernode as a single logical computer and embeds hardware-aware orchestration into the framework. Implemented in MindSpore, our HyperParallel architecture comprises HyperOffload for automated hierarchical memory management, HyperMPMD for fine-grained MPMD parallelism across heterogeneous workloads, and HyperShard for declarative parallel strategy specification. Together, these techniques significantly improve training and inference efficiency while reducing parallel programming and system tuning overhead, demonstrating the necessity of supernode affinity for next-generation AI frameworks.

翻译：大规模、稀疏、多模态及智能体式人工智能模型的兴起，正与硬件向超节点架构的演进同步。这种架构集成了数百至数千个加速器，具备超低延迟互连和统一内存池。然而，现有的人工智能框架并未针对此类架构进行高效设计，导致编程复杂度高、负载不均衡以及内存利用率低下。本文提出一种超节点亲和性人工智能框架，将超节点视为单一逻辑计算机，并将硬件感知的编排机制嵌入框架底层。我们在MindSpore中实现了超并行架构，其核心组件包括：用于自动化分级内存管理的HyperOffload、面向异构工作负载细粒度MPMD并行的HyperMPMD，以及支持声明式并行策略定义的HyperShard。这些技术共同显著提升了训练与推理效率，同时降低了并行编程与系统调优的开销，从而论证了超节点亲和性对于下一代人工智能框架的必要性。