While machine learning has advanced through massive parallelization, we identify a critical blind spot: some problems are fundamentally sequential. These "inherently serial" problems-from mathematical reasoning to physical simulations to sequential decision-making-require sequentially dependent computational steps that cannot be efficiently parallelized. We formalize this distinction in complexity theory, and demonstrate that current parallel-centric architectures face fundamental limitations on such tasks. Then, we show for first time that diffusion models despite their sequential nature are incapable of solving inherently serial problems. We argue that recognizing the serial nature of computation holds profound implications on machine learning, model design, and hardware development.

翻译：尽管机器学习通过大规模并行化取得了进展，但我们发现了一个关键盲点：某些问题本质上是串行的。这些“固有串行”问题——从数学推理到物理模拟再到序列决策——需要顺序依赖的计算步骤，无法被高效并行化。我们在计算复杂性理论中形式化了这一区分，并证明当前以并行为中心的体系结构在此类任务上存在根本性限制。随后，我们首次证明扩散模型尽管具有序列特性，仍无法解决固有串行问题。我们认为，认识计算的串行本质对机器学习、模型设计和硬件发展具有深远影响。