Communication overhead is a known bottleneck in federated learning (FL). To address this, lossy compression is commonly used on the information communicated between the server and clients during training. In horizontal FL, where each client holds a subset of the samples, such communication-compressed training methods have recently seen significant progress. However, in their vertical FL counterparts, where each client holds a subset of the features, our understanding remains limited. To address this, we propose an error feedback compressed vertical federated learning (EFVFL) method to train split neural networks. In contrast with previous communication-compressed methods for vertical FL, EFVFL does not require a vanishing compression error for the gradient norm to converge to zero for smooth nonconvex problems. By leveraging error feedback, our method can achieve a $\mathcal{O}(1/T)$ convergence rate in the full-batch case, improving over the state-of-the-art $\mathcal{O}(1/\sqrt{T})$ rate under $\mathcal{O}(1/\sqrt{T})$ compression error, and matching the rate of uncompressed methods. Further, when the objective function satisfies the Polyak-{\L}ojasiewicz inequality, our method converges linearly. In addition to improving convergence rates, our method also supports the use of private labels. Numerical experiments show that EFVFL significantly improves over the prior art, confirming our theoretical results.

翻译：通信开销是联邦学习（FL）中一个公认的瓶颈。为解决此问题，通常在训练过程中对服务器与客户端之间通信的信息采用有损压缩。在横向联邦学习中，每个客户端持有部分样本，此类通信压缩训练方法近年来已取得显著进展。然而，在纵向联邦学习（每个客户端持有部分特征）中，我们的理解仍然有限。为此，我们提出了一种基于误差反馈的压缩纵向联邦学习（EFVFL）方法来训练分割神经网络。与先前纵向联邦学习的通信压缩方法不同，对于光滑非凸问题，EFVFL不需要压缩误差趋近于零即可使梯度范数收敛到零。通过利用误差反馈，我们的方法在全批量情况下可实现$\mathcal{O}(1/T)$的收敛速度，优于在$\mathcal{O}(1/\sqrt{T})$压缩误差下的现有最佳$\mathcal{O}(1/\sqrt{T})$速度，并与未压缩方法的速度相匹配。此外，当目标函数满足Polyak-{\L}ojasiewicz不等式时，我们的方法可实现线性收敛。除了提升收敛速度外，我们的方法还支持使用私有标签。数值实验表明，EFVFL显著优于现有技术，验证了我们的理论结果。