We propose Flexible Vertical Federated Learning (Flex-VFL), a distributed machine algorithm that trains a smooth, non-convex function in a distributed system with vertically partitioned data. We consider a system with several parties that wish to collaboratively learn a global function. Each party holds a local dataset; the datasets have different features but share the same sample ID space. The parties are heterogeneous in nature: the parties' operating speeds, local model architectures, and optimizers may be different from one another and, further, they may change over time. To train a global model in such a system, Flex-VFL utilizes a form of parallel block coordinate descent, where parties train a partition of the global model via stochastic coordinate descent. We provide theoretical convergence analysis for Flex-VFL and show that the convergence rate is constrained by the party speeds and local optimizer parameters. We apply this analysis and extend our algorithm to adapt party learning rates in response to changing speeds and local optimizer parameters. Finally, we compare the convergence time of Flex-VFL against synchronous and asynchronous VFL algorithms, as well as illustrate the effectiveness of our adaptive extension.

翻译：我们提出灵活纵向联邦学习（Flex-VFL），一种在具有纵向划分数据的分布式系统中训练光滑非凸函数的分布式机器学习算法。我们考虑一个包含多个参与方希望协作学习全局函数的系统。每个参与方持有本地数据集；这些数据集具有不同特征但共享相同样本ID空间。参与方本质上是异构的：各参与方的运行速度、本地模型架构和优化器可能互不相同，且可能随时间动态变化。为在此类系统中训练全局模型，Flex-VFL采用并行块坐标下降方法，各参与方通过随机坐标下降训练全局模型的某个分区。我们为Flex-VFL提供理论收敛性分析，证明其收敛速率受参与方速度和本地优化器参数约束。基于该分析，我们扩展算法以根据参与方速度变化和本地优化器参数自适应调整学习率。最后，我们将Flex-VFL的收敛时间与同步和异步VFL算法进行对比，并验证了自适应扩展的有效性。