Federated learning systems typically allocate gradient compression by link speed. This is sensible when bandwidth and data informativeness align. However, under non-IID data, these signals often decorrelate or invert. A bandwidth-driven allocator then risks compressing the most informative gradients hardest. We propose HeteRo-Select, a framework that replaces bandwidth with a per-client informativeness score as the primary driver of compression. The score jointly governs three decisions per round: client selection, compression ratio, and server aggregation weight, with bandwidth retained only as a hard ceiling. Score-proportional selection provably reduces the effective heterogeneity of the chosen subset; score-proportional compression provably lowers aggregate top-$k$ error at fixed traffic. Under the exact FedCG simulation protocol, HeteRo-Select delivers a $1.78\times$ speedup and an $18.2\%$ reduction in traffic on CIFAR-10. The same configuration, unchanged, scales from a $7{,}850$-parameter logistic regression to an $11.27$M-parameter ResNet-18, hitting the accuracy target on three of four benchmarks. When bandwidth and informativeness are deliberately anti-correlated, the method still achieves the target accuracy with less traffic than the normal-bandwidth run.
翻译:联邦学习系统通常依据链路速度分配梯度压缩。当带宽与数据信息量一致时,这一策略具有合理性。然而在非独立同分布数据场景下,这两种信号经常出现去相关或反向关联现象,此时带宽驱动型的分配机制反而会倾向于对信息量最大的梯度实施最强压缩。本文提出HeteRo-Select框架,该框架采用客户端级信息量评分替代带宽作为压缩决策的核心驱动因素。该评分每轮联合调控三项决策:客户端选择、压缩比率和服务器聚合权重,而带宽仅作为硬性上限约束保留。理论证明,基于评分比例的客户端选择能有效降低所选子集的异质性程度;基于评分比例的压缩机制可在固定流量下降低聚合top-$k$误差。在标准FedCG仿真协议下,HeteRo-Select在CIFAR-10数据集上实现了$1.78\times$的加速比和$18.2\%$的流量缩减。相同配置未经调整即可从含$7{,}850$个参数的逻辑回归模型扩展至含$11.27$M个参数的ResNet-18网络,在四个基准测试中的三个达到精度目标。当带宽与信息量被人为反向关联时,该方法仍能以低于正常带宽运行的流量达到目标精度。