While machine learning has advanced through massive parallelization, we identify a critical blind spot: some problems are fundamentally sequential. These "inherently serial" problems-from mathematical reasoning to physical simulations to sequential decision-making-require sequentially dependent computational steps that cannot be efficiently parallelized. We formalize this distinction in complexity theory, and demonstrate that current parallel-centric architectures face fundamental limitations on such tasks. Then, we show for first time that diffusion models despite their sequential nature are incapable of solving inherently serial problems. We argue that recognizing the serial nature of computation holds profound implications on machine learning, model design, and hardware development.

翻译：尽管机器学习通过大规模并行化取得了进步，我们识别出一个关键的盲点：某些问题本质上是序列化的。这些“固有序列”问题——从数学推理到物理模拟再到序列决策——需要依赖顺序的计算步骤，无法高效并行化。我们在复杂性理论中正式化这一区分，并证明当前以并行为中心的架构在此类任务上存在根本性限制。接着，我们首次证明扩散模型尽管具有序列本质，却无法解决固有序列问题。我们认为，认识到计算的序列性质对机器学习、模型设计和硬件开发具有深远意义。