Federated Learning (FL) is a privacy-preserving paradigm, allowing edge devices to learn collaboratively without sharing data. Edge devices like Alexa and Siri are prospective sources of unlabeled audio data that can be tapped to learn robust audio representations. In this work, we bring Self-supervised Learning (SSL) and FL together to learn representations for Automatic Speech Recognition respecting data privacy constraints. We use the speaker and chapter information in the unlabeled speech dataset, Libri-Light, to simulate non-IID speaker-siloed data distributions and pre-train an LSTM encoder with the Contrastive Predictive Coding framework with FedSGD. We show that the pre-trained ASR encoder in FL performs as well as a centrally pre-trained model and produces an improvement of 12-15% (WER) compared to no pre-training. We further adapt the federated pre-trained models to a new language, French, and show a 20% (WER) improvement over no pre-training.

翻译：联邦学习（FL）是一种保护隐私的范式，允许边缘设备在不共享数据的情况下协同学习。像Alexa和Siri这样的边缘设备是未标注音频数据的潜在来源，可用于学习鲁棒的音频表征。在本研究中，我们将自监督学习（SSL）与联邦学习相结合，在遵守数据隐私约束的前提下，学习适用于自动语音识别的表征。我们利用未标注语音数据集Libri-Light中的说话者和章节信息，模拟非独立同分布（non-IID）的说话者隔离数据分布，并使用对比预测编码框架与FedSGD预训练一个LSTM编码器。实验表明，联邦学习下预训练的ASR编码器性能与集中式预训练模型相当，与无预训练相比，词错误率（WER）降低了12%-15%。我们进一步将联邦预训练模型迁移至新语言——法语，结果显示与无预训练相比，词错误率（WER）降低了20%。