Stochastic optimization algorithms have been successfully applied in several domains to find optimal solutions. Because of the ever-growing complexity of the integrated systems, novel stochastic algorithms are being proposed, which makes the task of the performance analysis of the algorithms extremely important. In this paper, we provide a novel ranking scheme to rank the algorithms over multiple single-objective optimization problems. The results of the algorithms are compared using a robust bootstrapping-based hypothesis testing procedure that is based on the principles of severity. Analogous to the football league scoring scheme, we propose pairwise comparison of algorithms as in league competition. Each algorithm accumulates points and a performance metric of how good or bad it performed against other algorithms analogous to goal differences metric in football league scoring system. The goal differences performance metric can not only be used as a tie-breaker but also be used to obtain a quantitative performance of each algorithm. The key novelty of the proposed ranking scheme is that it takes into account the performance of each algorithm considering the magnitude of the achieved performance improvement along with its practical relevance and does not have any distributional assumptions. The proposed ranking scheme is compared to classical hypothesis testing and the analysis of the results shows that the results are comparable and our proposed ranking showcases many additional benefits.

翻译：随机优化算法已在多个领域成功应用于寻找最优解。随着集成系统日益复杂，新型随机算法不断涌现，这使得算法性能分析任务变得至关重要。本文提出了一种新颖的排序方案，用于在多类单目标优化问题上对算法进行排序。算法结果的比较采用基于严重性原则的稳健自助法假设检验程序。类比足球联赛积分体系，我们提出类似联赛竞争的算法两两比较机制。每个算法通过与其他算法的对抗积累积分，并采用类似足球联赛净胜球指标的绩效度量来评估其表现优劣。这种净胜球绩效度量不仅可作为平局决胜依据，还能用于获取各算法的量化性能表现。该排序方案的核心创新在于：在考虑算法性能时，既关注所获性能提升的幅度，又兼顾其实际相关性，且无需任何分布假设。通过与经典假设检验方法的对比分析表明，所提排序方案结果具有可比性，且展现出诸多附加优势。