Federated learning (FL) achieves collaborative learning without the need for data sharing, thus preventing privacy leakage. To extend FL into a fully decentralized algorithm, researchers have applied distributed optimization algorithms to FL by considering machine learning (ML) tasks as parameter optimization problems. Conversely, the consensus-based multi-hop federated distillation (CMFD) proposed in the authors' previous work makes neural network (NN) models get close with others in a function space rather than in a parameter space. Hence, this study solves two unresolved challenges of CMFD: (1) communication cost reduction and (2) visualization of model convergence. Based on a proposed dynamic communication cost reduction method (DCCR), the amount of data transferred in a network is reduced; however, with a slight degradation in the prediction accuracy. In addition, a technique for visualizing the distance between the NN models in a function space is also proposed. The technique applies a dimensionality reduction technique by approximating infinite-dimensional functions as numerical vectors to visualize the trajectory of how the models change by the distributed learning algorithm.

翻译：联邦学习（FL）在不共享数据的前提下实现协同学习，从而防止隐私泄露。为将联邦学习扩展为完全去中心化算法，研究者通过将机器学习任务视为参数优化问题，将分布式优化算法应用于联邦学习。与此相反，作者先前工作中提出的基于共识的多跳联邦蒸馏（CMFD）使神经网络模型在函数空间而非参数空间中相互接近。因此，本研究解决了CMFD的两个未决挑战：（1）通信成本降低与（2）模型收敛性可视化。基于所提出的动态通信成本降低方法（DCCR），网络中传输的数据量得以减少，但预测精度略有下降。此外，还提出了一种在函数空间中可视化神经网络模型间距离的技术。该技术通过将无限维函数近似为数值向量来应用降维方法，从而可视化分布式学习算法如何改变模型的轨迹。