Many recent studies integrate federated learning (FL) with self-supervised learning (SSL) to take advantage of raw training data distributed across edge devices. However, edge devices often struggle with high computation and communication costs imposed by SSL and FL algorithms. To tackle this hindrance, we propose LW-FedSSL, a layer-wise federated self-supervised learning approach that allows edge devices to incrementally train one layer of the model at a time. LW-FedSSL comprises server-side calibration and representation alignment mechanisms to maintain comparable performance with end-to-end FedSSL while significantly lowering clients' resource requirements. The server-side calibration mechanism takes advantage of the resource-rich server in an FL environment to assist in global model training. Meanwhile, the representation alignment mechanism encourages closeness between representations of FL local models and those of the global model. Our experiments show that LW-FedSSL has a $3.3 \times$ lower memory requirement and a $3.2 \times$ cheaper communication cost than its end-to-end counterpart. We also explore a progressive training strategy called Prog-FedSSL that outperforms end-to-end training with a similar memory requirement and a $1.8 \times$ cheaper communication cost.

翻译：许多近期研究将联邦学习（FL）与自监督学习（SSL）相结合，以利用分布在边缘设备上的原始训练数据。然而，边缘设备往往难以承受SSL和FL算法带来的高计算与通信成本。为解决这一障碍，我们提出LW-FedSSL，一种逐层联邦自监督学习方法，允许边缘设备每次增量训练模型的一层。LW-FedSSL包含服务器端校准与表示对齐机制，在显著降低客户端资源需求的同时，保持与端到端FedSSL相当的性能。服务器端校准机制利用联邦学习环境中资源丰富的服务器辅助全局模型训练，而表示对齐机制则促进联邦学习局部模型与全局模型表示之间的接近性。实验表明，LW-FedSSL的内存需求比端到端方法低3.3倍，通信成本低3.2倍。此外，我们探索了一种渐进训练策略Prog-FedSSL，该策略在相似内存需求下优于端到端训练，且通信成本低1.8倍。