This paper presents a historical and technical survey of the hardware architectures, interconnection networks, and synchronization primitives that have shaped massively parallel systems over the past four decades. We examine the design of the NYU Ultracomputer and the IBM Research Parallel Processor Prototype (RP3), focusing on the hardware implementation of the Fetch-and-Add primitive in multistage interconnection networks. We contrast these early attempts at fine-grained, shared-memory hardware combining with the distributed-memory architectures of the IBM SP series and the modern in-network computation models found in NVIDIA SHARP and HPE Slingshot. We provide a technical analysis of message-passing synchronization, presenting a complete profiling of MPI operation frequencies and detailing the low-level hardware mapping of one-sided RMA atomics to PCIe Atomics and GPU caches. We investigate the software-hardware boundary in modern deep learning, detailing how HIP translation, Triton compilation, and 4-bit quantization (W4A16) execute on modern heterogeneous silicon. To evaluate alternative network node designs, we present a historical hardware case study analyzing the feasibility of implementing active combining switches using message-passing Inmos Transputers programmed in Occam. Finally, we contextualize the evolution of concurrent software synchronization by examining Isaac Dimitrovsky's parallel "group lock" primitive, tracing its downstream echoes in group mutual exclusion (GME) and room synchronization, and reflect on the historical, philosophical divide between American systems engineering and European formal methods.

翻译：本文对过去四十年塑造大规模并行系统的硬件架构、互连网络及同步原语进行了历史与技术综述。我们考察了NYU超级计算机与IBM研究并行处理器原型（RP3）的设计，重点关注多级互连网络中“取并加”原语的硬件实现。将这些早期细粒度共享内存硬件组合尝试，与IBM SP系列的分布式内存架构、以及NVIDIA SHARP和HPE Slingshot中的现代网络内计算模型进行对比。我们提供了消息传递同步的技术分析，完整呈现MPI操作频率分布，并详细阐述单边RMA原子操作到PCIe原子操作与GPU缓存的低层硬件映射。探究现代深度学习中的软硬件边界，详细说明HIP转换、Triton编译及4位量化（W4A16）如何在现代异构芯片上执行。为评估替代网络节点设计，我们呈现一个历史硬件案例研究，通过分析使用Occam语言编程的基于消息传递的Inmos Transputer实现主动组合交换机的可行性。最后，通过考察Isaac Dimitrovsky的并行"组锁"原语，追溯其在组互斥（GME）与房间同步中的下游回声，结合美国系统工程与欧洲形式化方法之间的历史哲学分野，对并发软件同步的演进进行历史性解读。