In machine learning (ML), a widespread adage is that the area under the precision-recall curve (AUPRC) is a superior metric for model comparison to the area under the receiver operating characteristic (AUROC) for binary classification tasks with class imbalance. This paper challenges this notion through novel mathematical analysis, illustrating that AUROC and AUPRC can be concisely related in probabilistic terms. We demonstrate that AUPRC, contrary to popular belief, is not superior in cases of class imbalance and might even be a harmful metric, given its inclination to unduly favor model improvements in subpopulations with more frequent positive labels. This bias can inadvertently heighten algorithmic disparities. Prompted by these insights, a thorough review of existing ML literature was conducted, utilizing large language models to analyze over 1.5 million papers from arXiv. Our investigation focused on the prevalence and substantiation of the purported AUPRC superiority. The results expose a significant deficit in empirical backing and a trend of misattributions that have fuelled the widespread acceptance of AUPRC's supposed advantages. Our findings represent a dual contribution: a significant technical advancement in understanding metric behaviors and a stark warning about unchecked assumptions in the ML community. All experiments are accessible at https://github.com/mmcdermott/AUC_is_all_you_need.

翻译：在机器学习领域，普遍认为对于存在类别不平衡的二分类任务，相较于受试者工作特征曲线下面积（AUROC），精确率-召回率曲线下面积（AUPRC）是更优的模型比较指标。本文通过创新性数学分析挑战了这一观点，以概率论术语简明阐释了AUROC与AUPRC之间的关联。我们证明，与普遍认知相反，AUPRC在类别不平衡场景下并不具有优越性，甚至可能成为有害指标——因其倾向于过度奖励正标签更频繁的亚群体中模型性能的改进。这种偏差可能间接加剧算法的不公平性。基于上述发现，我们利用大型语言模型对arXiv上逾150万篇论文进行了系统性文献综述，重点考察AUPRC所谓优越性的普及程度与实证依据。结果显示，该观点存在显著的实证支撑缺失，且一系列错误归因现象助长了AUPRC优势论的广泛传播。本研究具有双重贡献：既是对指标行为认知的重要技术突破，也是对机器学习社区中未经验证假设的严厉警示。所有实验数据与代码均可在https://github.com/mmcdermott/AUC_is_all_you_need获取。