In machine learning (ML), a widespread adage is that the area under the precision-recall curve (AUPRC) is a superior metric for model comparison to the area under the receiver operating characteristic (AUROC) for binary classification tasks with class imbalance. This paper challenges this notion through novel mathematical analysis, illustrating that AUROC and AUPRC can be concisely related in probabilistic terms. We demonstrate that AUPRC, contrary to popular belief, is not superior in cases of class imbalance and might even be a harmful metric, given its inclination to unduly favor model improvements in subpopulations with more frequent positive labels. This bias can inadvertently heighten algorithmic disparities. Prompted by these insights, a thorough review of existing ML literature was conducted, utilizing large language models to analyze over 1.5 million papers from arXiv. Our investigation focused on the prevalence and substantiation of the purported AUPRC superiority. The results expose a significant deficit in empirical backing and a trend of misattributions that have fuelled the widespread acceptance of AUPRC's supposed advantages. Our findings represent a dual contribution: a significant technical advancement in understanding metric behaviors and a stark warning about unchecked assumptions in the ML community. All experiments are accessible at https://github.com/mmcdermott/AUC_is_all_you_need.

翻译：在机器学习领域，一个广为流传的观点认为，对于存在类别不平衡的二分类任务，精确率-召回率曲线下面积（AUPRC）是比受试者工作特征曲线下面积（AUROC）更优越的模型比较指标。本文通过新颖的数学分析对这一观点提出质疑，揭示了AUROC与AUPRC在概率意义上可被简洁关联。我们证明，AUPRC与普遍认知相反，在类别不平衡情况下并非更优，甚至可能成为一个有害指标——因其倾向于不合理地偏好正标签更频繁的子群体中的模型改进，这种偏差会无意中加剧算法歧视。基于这些发现，我们利用大语言模型对arXiv上超过150万篇论文进行了系统性文献综述，重点关注所谓AUPRC优越性的普遍程度与实证依据。结果暴露了该主张在实证支持方面的显著缺失，以及推动AUPRC所谓优势广泛接受的一系列归因偏差。本研究的贡献具有双重性：既是对指标行为理解的重大技术突破，也是对机器学习社区中未经检验的假设提出的严厉警示。所有实验数据与代码均可通过https://github.com/mmcdermott/AUC_is_all_you_need获取。