This paper introduces the award-winning deep learning (DL) library called LibAUC for implementing state-of-the-art algorithms towards optimizing a family of risk functions named X-risks. X-risks refer to a family of compositional functions in which the loss function of each data point is defined in a way that contrasts the data point with a large number of others. They have broad applications in AI for solving classical and emerging problems, including but not limited to classification for imbalanced data (CID), learning to rank (LTR), and contrastive learning of representations (CLR). The motivation of developing LibAUC is to address the convergence issues of existing libraries for solving these problems. In particular, existing libraries may not converge or require very large mini-batch sizes in order to attain good performance for these problems, due to the usage of the standard mini-batch technique in the empirical risk minimization (ERM) framework. Our library is for deep X-risk optimization (DXO) that has achieved great success in solving a variety of tasks for CID, LTR and CLR. The contributions of this paper include: (1) It introduces a new mini-batch based pipeline for implementing DXO algorithms, which differs from existing DL pipeline in the design of controlled data samplers and dynamic mini-batch losses; (2) It provides extensive benchmarking experiments for ablation studies and comparison with existing libraries. The LibAUC library features scalable performance for millions of items to be contrasted, faster and better convergence than existing libraries for optimizing X-risks, seamless PyTorch deployment and versatile APIs for various loss optimization. Our library is available to the open source community at https://github.com/Optimization-AI/LibAUC, to facilitate further academic research and industrial applications.

翻译：本文介绍了获奖的深度学习库LibAUC，该库用于实现针对一类名为X-risks的风险函数族的最先进算法。X-risks指一类组合函数，其中每个数据点的损失函数定义为将该数据点与大量其他数据点进行对比。它们在人工智能领域具有广泛应用，可用于解决包括但不限于不平衡数据分类（CID）、学习排序（LTR）和对比表示学习（CLR）等经典及新兴问题。开发LibAUC的动机旨在解决现有库在求解这些问题时存在的收敛性问题。具体而言，由于在经验风险最小化（ERM）框架中使用标准小批量技术，现有库可能无法收敛或需要非常大的小批量大小才能在这些问题上获得良好性能。我们的库专用于深度X风险优化（DXO），已在解决CID、LTR和CLR的多种任务中取得巨大成功。本文贡献包括：（1）引入了一种基于小批量的新流水线来执行DXO算法，该流水线在受控数据采样器和动态小批量损失的设计上不同于现有深度学习流水线；（2）提供了广泛的基准实验用于消融研究及与现有库的比较。LibAUC库具有以下特点：可对百万级项目进行可扩展性对比、比现有库更快且更优的X-risks优化收敛速度、与PyTorch的无缝部署以及支持多种损失优化的通用API。我们的库已在https://github.com/Optimization-AI/LibAUC上向开源社区开放，以促进进一步的学术研究和工业应用。