Traditionally, massively parallel applications are executed on distributed systems, where computing nodes are distant enough that the parallelization schemes must minimize communication and synchronization to achieve scalability. Mapping communication-intensive workloads to distributed systems requires complicated problem partitioning and dataset pre-processing. With the current AI-driven trend of having thousands of interconnected processors per chip, there is an opportunity to re-think these communication-bottlenecked workloads. This bottleneck often arises from data structure traversals, which cause irregular memory accesses and poor cache locality. Recent works have introduced task-based parallelization schemes to accelerate graph traversal and other sparse workloads. Data structure traversals are split into tasks and pipelined across processing units (PUs). Dalorex demonstrated the highest scalability (up to thousands of PUs on a single chip) by having the entire dataset on-chip, scattered across PUs, and executing the tasks at the PU where the data is local. However, it also raised questions on how to scale to larger datasets when all the memory is on chip, and at what cost. To address these challenges, we propose a scalable architecture composed of a grid of Data-Centric Reconfigurable Array (DCRA) chiplets. Package-time reconfiguration enables creating chip products that optimize for different target metrics, such as time-to-solution, energy, or cost, while software reconfigurations avoid network saturation when scaling to millions of PUs across many chip packages. We evaluate six applications and four datasets, with several configurations and memory technologies, to provide a detailed analysis of the performance, power, and cost of data-local execution at scale. Our parallelization of Breadth-First-Search with RMAT-26 across a million PUs reaches 3323 GTEPS.

翻译：传统上，大规模并行应用运行于分布式系统中，由于计算节点间距离较远，并行化方案必须最小化通信与同步以实现可扩展性。将通信密集型工作负载映射到分布式系统需要复杂的问题划分和数据集预处理。在当前人工智能驱动的趋势下，每颗芯片集成数千个互连处理器，这为重新思考这些受通信瓶颈制约的工作负载提供了机遇。此类瓶颈常源于数据结构遍历，这会导致不规则的内存访问和较差的缓存局部性。近期研究引入了基于任务的并行化方案来加速图遍历及其他稀疏型工作负载。数据结构遍历被拆分为任务，并在处理单元之间进行流水线化。Dalorex通过将整个数据集存放在芯片上、分散至各个处理单元，并在数据所在处理单元执行任务，展示了最高可扩展性（单芯片上支持数千个处理单元）。然而，这也引发了关于如何在全内存片上场景下扩展至更大数据集，以及相应代价的问题。为应对这些挑战，我们提出一种可扩展架构，由网格化数据中心型可重构阵列芯片组构成。封装时重构使得我们能够创建针对不同目标指标（如求解时间、能耗或成本）进行优化的芯片产品，而软件重构则能在跨多芯片封装扩展到数百万个处理单元时避免网络饱和。我们评估了六种应用和四个数据集，结合多种配置与内存技术，对数据本地化执行在大规模场景下的性能、功耗与成本进行了详细分析。采用RMAT-26数据集实现的广度优先搜索并行化方案，在百万处理单元规模下达到了3323 GTEPS。