Federated learning encounters substantial challenges with heterogeneous data, leading to performance degradation and convergence issues. While considerable progress has been achieved in mitigating such an impact, the reliability aspect of federated models has been largely disregarded. In this study, we conduct extensive experiments to investigate the reliability of both generic and personalized federated models. Our exploration uncovers a significant finding: \textbf{federated models exhibit unreliability when faced with heterogeneous data}, demonstrating poor calibration on in-distribution test data and low uncertainty levels on out-of-distribution data. This unreliability is primarily attributed to the presence of biased projection heads, which introduce miscalibration into the federated models. Inspired by this observation, we propose the "Assembled Projection Heads" (APH) method for enhancing the reliability of federated models. By treating the existing projection head parameters as priors, APH randomly samples multiple initialized parameters of projection heads from the prior and further performs targeted fine-tuning on locally available data under varying learning rates. Such a head ensemble introduces parameter diversity into the deterministic model, eliminating the bias and producing reliable predictions via head averaging. We evaluate the effectiveness of the proposed APH method across three prominent federated benchmarks. Experimental results validate the efficacy of APH in model calibration and uncertainty estimation. Notably, APH can be seamlessly integrated into various federated approaches but only requires less than 30\% additional computation cost for 100$\times$ inferences within large models.

翻译：联邦学习在面对数据异构时面临显著挑战，导致性能下降和收敛问题。尽管在缓解此类影响方面已取得重大进展，但联邦模型的可靠性层面在很大程度上被忽视。在本研究中，我们通过大量实验探究通用型和个性化联邦模型的可靠性。我们的探索揭示了一个重要发现：**联邦模型在面临异构数据时表现出不可靠性**，即在分布内测试数据上校准不良，并在分布外数据上呈现低不确定性水平。这种不可靠性主要归因于有偏投影头的存在，这些投影头将错误校准引入联邦模型。受此观察启发，我们提出“组装投影头”（APH）方法以增强联邦模型的可靠性。通过将现有投影头参数视为先验，APH从该先验中随机采样多个投影头的初始化参数，并在不同学习率下对本地可用数据进行针对性微调。这种头部集成向确定性模型引入参数多样性，消除偏差并通过头部平均产生可靠预测。我们在三个主流联邦基准上评估了所提APH方法的有效性。实验结果验证了APH在模型校准和不确定性估计方面的功效。值得注意的是，APH可无缝集成到多种联邦方法中，但仅需在大规模模型中执行100次推理时增加不到30%的额外计算成本。