Pipeline parallelism is a key technique for distributed training of large language models because it reduces per-device parameter and activation memory. However, comparing pipeline schedules is difficult: analytical models expose structural quantities such as bubble ratios, while end-to-end hardware experiments are costly and system-specific. In this work, we introduce a tabular schedule abstraction and a unified multi-abstraction methodology that connects formula-based reasoning, idealized schedule tables, and communication-aware execution simulation. Using this framework, we compare GPipe, 1F1B, Chimera, and Hanayo in its restricted regime across multiple modeled system configurations. Our results show that schedule rankings are not abstraction-invariant: communication can negate structural advantages suggested by bubble analysis alone. Under the assumptions considered here, GPipe and 1F1B are runtime-equivalent, but 1F1B achieves a lower activation-memory peak. Chimera is advantageous mainly at low microbatch counts and in communication-favorable regimes, while Hanayo is effective in its intended restricted operating point but remains sensitive to network bottlenecks. We further study an asymmetric Chimera-style placement, which does not reduce the global peak memory requirement but reveals limited runtime gains in shallow pipelines. Overall, pipeline schedule quality is meaningful only in the context of the modeled execution environment.

翻译：流水线并行性是大型语言模型分布式训练的关键技术，因为它能降低每设备的参数和激活内存。然而，比较流水线调度方案颇为困难：分析模型暴露了诸如气泡比率等结构量，而端到端硬件实验成本高昂且依赖特定系统。在这项工作中，我们引入了一种表格调度抽象和统一的多元抽象方法论，该方法将基于公式的推理、理想化的调度表格以及通信感知的执行模拟联系起来。利用该框架，我们在多种建模系统配置下比较了GPipe、1F1B、Chimera和Hanayo（在其受限体制内）。我们的结果表明，调度方案的排名并非抽象不变：通信可以抵消仅由气泡分析所揭示的结构优势。在本文考虑假设下，GPipe和1F1B在运行时是等效的，但1F1B实现了更低的激活内存峰值。Chimera的优势主要体现在低微批次数和通信友好的场景中，而Hanayo在其预定的受限运行点有效，但仍对网络瓶颈敏感。我们进一步研究了一种非对称的类Chimera放置方式，该方式并未降低全局峰值内存需求，但在浅层流水线中显示出有限的运行时收益。总体而言，流水线调度质量仅在其建模的执行环境背景下才有意义。