Language model training in distributed settings is limited by the communication cost of gradient exchanges. In this short note, we extend recent work from Malladi et al. (2023), using shared randomness to perform distributed fine-tuning with low bandwidth. The method is a natural decentralized extension of memory-efficient Simultaneous Perturbation Stochastic Approximation (SPSA). Each iteration, each machine seeds a Random Number Generator (RNG) to perform local reproducible perturbations on model weights and calculate and exchange scalar projected gradients, which are then used to update each model. By using a (machine, sample) identifier as the random seed, each model can regenerate one another's perturbations. As machines only exchange single-byte projected gradients, this is highly communication efficient. There are also potential privacy benefits, as projected gradients may be calculated on different training data, and models never access the other's data. Our approach not only drastically reduces communication bandwidth requirements but also accommodates dynamic addition or removal of machines during the training process and retains the memory-efficient and inference-only advantages of recent work. We perform proof-of-concept experiments to demonstrate the potential usefulness of this method, building off of rich literature on distributed optimization and memory-efficient training.

翻译：分布式场景下的语言模型训练受限于梯度交换的通信成本。本短文扩展了Malladi等人（2023）的最新工作，利用共享随机化实现低带宽的分布式微调。该方法是对内存高效同步扰动随机逼近的自然去中心化扩展。每次迭代中，每台机器对随机数生成器进行播种，对模型权重执行局部可复现扰动，计算并交换标量投影梯度，随后用于更新各自模型。通过采用（机器，样本）标识符作为随机种子，各模型可彼此复现对方的扰动。由于机器间仅交换单字节投影梯度，该方法具有极高的通信效率。同时存在潜在隐私优势：投影梯度可在不同训练数据上计算，且各模型从未访问他人数据。我们的方法不仅大幅降低通信带宽需求，还能在训练过程中动态适应机器的增删，并保留了最新工作的内存高效与推理专用优势。我们基于分布式优化与内存高效训练的丰富文献开展概念验证实验，证明该方法的潜在实用性。