With the ever-widening spread of the Internet of Things (IoT) and Edge Computing paradigms, centralized Machine and Deep Learning (ML/DL) have become challenging due to existing distributed data silos containing sensitive information. The rising concern for data privacy is promoting the development of collaborative and privacy-preserving ML/DL techniques such as Federated Learning (FL). FL enables data privacy by design since the local data of participants are not exposed during the creation of the global and collaborative model. However, data privacy and performance are no longer sufficient, and there is a real necessity to trust model predictions. The literature has proposed some works on trustworthy ML/DL (without data privacy), where robustness, fairness, explainability, and accountability are identified as relevant pillars. However, more efforts are needed to identify trustworthiness pillars and evaluation metrics relevant to FL models and to create solutions computing the trustworthiness level of FL models. Thus, this work analyzes the existing requirements for trustworthiness evaluation in FL and proposes a comprehensive taxonomy of six pillars (privacy, robustness, fairness, explainability, accountability, and federation) with notions and more than 30 metrics for computing the trustworthiness of FL models. Then, an algorithm called FederatedTrust has been designed according to the pillars and metrics identified in the previous taxonomy to compute the trustworthiness score of FL models. A prototype of FederatedTrust has been implemented and deployed into the learning process of FederatedScope, a well-known FL framework. Finally, four experiments performed with different configurations of FederatedScope using the FEMNIST dataset under different federation configurations demonstrated the usefulness of FederatedTrust when computing the trustworthiness of FL models.

翻译：随着物联网和边缘计算范式的日益普及，集中式机器学习和深度学习因存在包含敏感信息的分布式数据孤岛而面临挑战。对数据隐私日益增长的关切正在推动联邦学习等协作式隐私保护机器学习和深度学习技术的发展。联邦学习通过设计保障数据隐私，因为在创建全局协作模型的过程中，参与者的本地数据不会被暴露。然而，数据隐私和性能已不再足够，对模型预测的信任已成为实际需求。现有文献提出了一些关于可信机器学习和深度学习（不涉及数据隐私）的研究，其中鲁棒性、公平性、可解释性和可问责性被认定为关键支柱。但进一步的努力仍需用于识别与联邦学习模型相关的可信支柱和评估指标，并创建计算联邦学习模型可信水平的解决方案。因此，本研究分析了联邦学习中可信评估的现有需求，提出了一个包含六大支柱（隐私性、鲁棒性、公平性、可解释性、可问责性和联邦性）的全面分类体系，并给出了概念及30多个用于计算联邦学习模型可信度的指标。随后，根据上述分类体系中的支柱和指标，设计了一种名为FederatedTrust的算法，用于计算联邦学习模型的可信度评分。我们实现了FederatedTrust的原型，并将其部署到著名联邦学习框架FederatedScope的学习过程中。最后，通过使用FEMNIST数据集在不同联邦配置下进行的四项实验（采用不同FederatedScope配置），验证了FederatedTrust在计算联邦学习模型可信度时的有效性。