Topic modeling has emerged as a valuable tool for discovering patterns and topics within large collections of documents. However, when cross-analysis involves multiple parties, data privacy becomes a critical concern. Federated topic modeling has been developed to address this issue, allowing multiple parties to jointly train models while protecting pri-vacy. However, there are communication and performance challenges in the federated sce-nario. In order to solve the above problems, this paper proposes a method to establish a federated topic model while ensuring the privacy of each node, and use neural network model pruning to accelerate the model, where the client periodically sends the model neu-ron cumulative gradients and model weights to the server, and the server prunes the model. To address different requirements, two different methods are proposed to determine the model pruning rate. The first method involves slow pruning throughout the entire model training process, which has limited acceleration effect on the model training process, but can ensure that the pruned model achieves higher accuracy. This can significantly reduce the model inference time during the inference process. The second strategy is to quickly reach the target pruning rate in the early stage of model training in order to accelerate the model training speed, and then continue to train the model with a smaller model size after reaching the target pruning rate. This approach may lose more useful information but can complete the model training faster. Experimental results show that the federated topic model pruning based on the variational autoencoder proposed in this paper can greatly accelerate the model training speed while ensuring the model's performance.

翻译：主题建模已成为从大规模文档集合中发现模式和主题的有价值工具。然而，当跨分析涉及多方时，数据隐私成为关键问题。联邦主题建模为解决此问题而发展起来，允许多方在保护隐私的同时联合训练模型。但联邦场景中存在通信和性能方面的挑战。为解决上述问题，本文提出了一种方法，在确保各节点隐私的前提下建立联邦主题模型，并通过神经网络模型剪枝加速模型，其中客户端定期向服务器发送模型神经元累积梯度和模型权重，由服务器执行模型剪枝。针对不同需求，提出了两种确定模型剪枝率的方法。第一种方法是在整个模型训练过程中进行缓慢剪枝，该方法对模型训练过程的加速效果有限，但能确保剪枝后的模型达到较高精度，可显著减少推理过程中的模型推理时间。第二种策略是在模型训练早期快速达到目标剪枝率以加速模型训练速度，然后在达到目标剪枝率后以更小的模型规模继续训练。该方法可能丢失更多有用信息，但能更快完成模型训练。实验结果表明，本文提出的基于变分自编码器的联邦主题模型剪枝方法能在保证模型性能的同时大幅加速模型训练速度。