This paper leverages the statistics of extreme values to predict the worst-case convergence times of machine learning algorithms. Timing is a critical non-functional property of ML systems, and providing the worst-case converge times is essential to guarantee the availability of ML and its services. However, timing properties such as worst-case convergence times (WCCT) are difficult to verify since (1) they are not encoded in the syntax or semantics of underlying programming languages of AI, (2) their evaluations depend on both algorithmic implementations and underlying systems, and (3) their measurements involve uncertainty and noise. Therefore, prevalent formal methods and statistical models fail to provide rich information on the amounts and likelihood of WCCT. Our key observation is that the timing information we seek represents the extreme tail of execution times. Therefore, extreme value theory (EVT), a statistical discipline that focuses on understanding and predicting the distribution of extreme values in the tail of outcomes, provides an ideal framework to model and analyze WCCT in the training and inference phases of ML paradigm. Building upon the mathematical tools from EVT, we propose a practical framework to predict the worst-case timing properties of ML. Over a set of linear ML training algorithms, we show that EVT achieves a better accuracy for predicting WCCTs than relevant statistical methods such as the Bayesian factor. On the set of larger machine learning training algorithms and deep neural network inference, we show the feasibility and usefulness of EVT models to accurately predict WCCTs, their expected return periods, and their likelihood.

翻译：本文利用极值统计学方法预测机器学习算法的最坏情况收敛时间。时序特性是机器学习系统的关键非功能属性，提供最坏情况收敛时间对于保障机器学习及其服务的可用性至关重要。然而，最坏情况收敛时间（WCCT）等时序特性难以验证，原因在于：（1）这些特性并未编码在人工智能底层编程语言的语法或语义中；（2）其评估依赖于算法实现与底层系统的共同作用；（3）其测量存在不确定性与噪声干扰。因此，现有形式化方法与统计模型无法提供关于WCCT数值及发生概率的丰富信息。本文的核心发现是：所需时序信息对应执行时间的极端尾部。极值理论（EVT）作为专注于理解和预测结果尾部极值分布规律的统计学科，为建模和分析机器学习范式中训练与推理阶段的WCCT提供了理想框架。基于极值理论的数学工具，我们提出预测机器学习最坏情况时序特性的实用框架。在线性机器学习训练算法集合上，我们证明极值理论在预测WCCT精度上优于贝叶斯因子等相关统计方法。在较大规模机器学习训练算法与深度神经网络推理场景中，我们验证了极值理论模型在准确预测WCCT、预期重现周期及其发生概率方面的可行性与实用性。