In traditional machine learning, it is trivial to conduct model evaluation since all data samples are managed centrally by a server. However, model evaluation becomes a challenging problem in federated learning (FL), which is called federated evaluation in this work. This is because clients do not expose their original data to preserve data privacy. Federated evaluation plays a vital role in client selection, incentive mechanism design, malicious attack detection, etc. In this paper, we provide the first comprehensive survey of existing federated evaluation methods. Moreover, we explore various applications of federated evaluation for enhancing FL performance and finally present future research directions by envisioning some challenges.

翻译：在传统机器学习中，模型评估是轻而易举的，因为所有数据样本均由服务器集中管理。然而，在联邦学习（FL）中，模型评估成为一个具有挑战性的问题，本文将其称为联邦评估。这是因为客户端为了保障数据隐私，不会公开其原始数据。联邦评估在客户端选择、激励机制设计、恶意攻击检测等方面发挥着至关重要的作用。本文首次对现有的联邦评估方法进行了全面综述。此外，我们探讨了联邦评估在提升FL性能中的多种应用，并最终通过展望若干挑战，阐述了未来的研究方向。