In traditional machine learning, it is trivial to conduct model evaluation since all data samples are managed centrally by a server. However, model evaluation becomes a challenging problem in federated learning (FL), which is called federated evaluation in this work. This is because clients do not expose their original data to preserve data privacy. Federated evaluation plays a vital role in client selection, incentive mechanism design, malicious attack detection, etc. In this paper, we provide the first comprehensive survey of existing federated evaluation methods. Moreover, we explore various applications of federated evaluation for enhancing FL performance and finally present future research directions by envisioning some challenges.

翻译：在传统机器学习中，由于所有数据样本由服务器集中管理，模型评估易于实现。然而，在联邦学习（FL）中，模型评估成为一个具有挑战性的问题，本文将其称为联邦评估。这是因为客户端为保护数据隐私不会暴露其原始数据。联邦评估在客户端选择、激励机制设计、恶意攻击检测等方面发挥着至关重要的作用。本文首次对现有联邦评估方法进行了全面综述，同时探讨了联邦评估在提升FL性能方面的多种应用，最后通过展望当前面临的挑战提出了未来研究方向。