We address two major obstacles to practical use of supervised classifiers on distributed private data. Whether a classifier was trained by a federation of cooperating clients or trained centrally out of distribution, (1) the output scores must be calibrated, and (2) performance metrics must be evaluated -- all without assembling labels in one place. In particular, we show how to perform calibration and compute precision, recall, accuracy and ROC-AUC in the federated setting under three privacy models (i) secure aggregation, (ii) distributed differential privacy, (iii) local differential privacy. Our theorems and experiments clarify tradeoffs between privacy, accuracy, and data efficiency. They also help decide whether a given application has sufficient data to support federated calibration and evaluation.

翻译：我们解决了在分配的私人数据上实际使用受监督分类员的两大障碍:分类员是否接受过合作客户联合会的培训或集中培训,是否在分配过程中得到集中培训;(1)产出分数必须校准;(2)性能指标必须评估 -- -- 均不在一个地方组装标签;特别是,我们展示了如何在三个隐私模式下在联邦环境中进行校准和计算、回溯、准确性和ROC-AUC,(一) 安全聚合;(二) 分散的隐私;(三) 地方差异性隐私。我们的理论和实验澄清了隐私、准确性和数据效率之间的取舍。它们也有助于确定特定应用程序是否有足够的数据支持联合校准和评估。