This paper introduces an approach to employ clipped uniform quantization in federated learning settings, aiming to enhance model efficiency by reducing communication overhead without compromising accuracy. By employing optimal clipping thresholds and adaptive quantization schemes, our method significantly curtails the bit requirements for model weight transmissions between clients and the server. We explore the implications of symmetric clipping and uniform quantization on model performance, highlighting the utility of stochastic quantization to mitigate quantization artifacts and improve model robustness. Through extensive simulations on the MNIST dataset, our results demonstrate that the proposed method achieves near full-precision performance while ensuring substantial communication savings. Specifically, our approach facilitates efficient weight averaging based on quantization errors, effectively balancing the trade-off between communication efficiency and model accuracy. The comparative analysis with conventional quantization methods further confirms the superiority of our technique.
翻译:本文提出了一种在联邦学习环境中采用截断均匀量化的方法,旨在通过降低通信开销而不牺牲精度来提升模型效率。通过采用最优截断阈值和自适应量化方案,我们的方法显著减少了客户端与服务器之间模型权重传输所需的比特数。我们探讨了对称截断与均匀量化对模型性能的影响,强调了随机量化在减轻量化伪影和提升模型鲁棒性方面的效用。通过在MNIST数据集上的大量仿真实验,我们的结果表明,所提方法在确保显著节省通信成本的同时,实现了接近全精度的性能。具体而言,我们的方法基于量化误差实现了高效的权重平均,有效平衡了通信效率与模型精度之间的权衡。与传统量化方法的对比分析进一步证实了我们技术的优越性。