The growing scale of deep learning demands distributed training frameworks that jointly reason about parallelism, memory, and network topology. Prior works often rely on heuristic or topology-agnostic search, handling communication and memory separately. Without per-device memory awareness, these methods typically ensure feasibility post hoc by sharding parameters and activations across many devices, increasing synchronization, inflating communication, and underutilizing compute-limiting scalability and efficiency on real datacenter networks. We present NEST, a network-, compute-, and memory-aware device placement framework that unifies model parallelism, topology modeling, and memory feasibility via structured dynamic programming. NEST's DP operates on operator graphs with tensor and expert parallel configurations, explicit allreduce latencies across hierarchical or arbitrary networks, and memory/compute profiles. By factoring parallelism across tensor, pipeline, data, and expert dimensions, NEST defines a principled search space for hybrid strategies while jointly optimizing co-location, network latency, and memory feasibility. Evaluations across diverse hardware and networks show NEST achieves up to 2.43 times higher throughput, better memory efficiency, and improved scalability over state-of-the-art baselines, providing a foundation for co-designing parallelization strategies and datacenter interconnects for next-generation AI infrastructure. The source code of NEST is available at: https://github.com/scai-tech/Nest

翻译：深度学习规模的不断增长要求分布式训练框架能够协同考虑并行性、内存和网络拓扑。先前的研究通常依赖于启发式或拓扑无关的搜索方法，将通信与内存问题分开处理。由于缺乏对单设备内存的感知，这些方法通常通过将参数和激活张量分片到多个设备上来事后确保可行性，这增加了同步开销、扩大了通信量，并导致计算资源利用率不足，从而限制了在实际数据中心网络上的可扩展性和效率。本文提出NEST，一种集网络感知、计算感知与内存感知于一体的设备布局框架，它通过结构化动态规划统一了模型并行、拓扑建模和内存可行性分析。NEST的动态规划算法基于算子图运行，该图包含张量与专家并行配置、跨层次或任意网络的显式Allreduce延迟，以及内存/计算性能分析。通过将并行性分解为张量、流水线、数据和专家等多个维度，NEST为混合并行策略定义了一个结构化的搜索空间，同时联合优化设备共置、网络延迟和内存可行性。在不同硬件和网络环境下的评估表明，相较于现有先进基线方法，NEST能够实现高达2.43倍的吞吐量提升、更优的内存效率以及更好的可扩展性，为下一代人工智能基础设施中并行化策略与数据中心互连的协同设计提供了基础。NEST的源代码已公开于：https://github.com/scai-tech/Nest