Receiver Operating Characteristic (ROC) and Precision-Recall (PR) curves are fundamental tools for evaluating machine learning classifiers, offering detailed insights into the trade-offs between true positive rate vs. false positive rate (ROC) or precision vs. recall (PR). However, in Federated Learning (FL) scenarios, where data is distributed across multiple clients, computing these curves is challenging due to privacy and communication constraints. Specifically, the server cannot access raw prediction scores and class labels, which are used to compute the ROC and PR curves in a centralized setting. In this paper, we propose a novel method for approximating ROC and PR curves in a federated setting by estimating quantiles of the prediction score distribution under distributed differential privacy. We provide theoretical bounds on the Area Error (AE) between the true and estimated curves, demonstrating the trade-offs between approximation accuracy, privacy, and communication cost. Empirical results on real-world datasets demonstrate that our method achieves high approximation accuracy with minimal communication and strong privacy guarantees, making it practical for privacy-preserving model evaluation in federated systems.

翻译：接收者操作特征（ROC）曲线与精确率-召回率（PR）曲线是评估机器学习分类器的基本工具，能够深入揭示真阳性率与假阳性率（ROC）或精确率与召回率（PR）之间的权衡关系。然而，在联邦学习（FL）场景中，数据分布在多个客户端之间，由于隐私和通信限制，计算这些曲线具有挑战性。具体而言，服务器无法访问原始预测分数和类别标签，而这些在集中式设置中用于计算ROC和PR曲线。本文提出一种在联邦环境下近似计算ROC和PR曲线的新方法，通过估计分布式差分隐私约束下预测分数分布的分位数来实现。我们为真实曲线与估计曲线之间的面积误差（AE）提供了理论界，揭示了近似精度、隐私保护与通信成本之间的权衡关系。在真实数据集上的实验结果表明，我们的方法能以最小通信开销和强隐私保证实现高近似精度，使其适用于联邦系统中隐私保护的模型评估。