Hyperparameter optimization is crucial to achieving high performance in deep learning. On top of the performance, other criteria such as inference time or memory requirement often need to be optimized due to some practical reasons. This motivates research on multi-objective optimization (MOO). However, Pareto fronts of MOO methods are often shown without considering the variability caused by random seeds and this makes the performance stability evaluation difficult. Although there is a concept named empirical attainment surface to enable the visualization with uncertainty over multiple runs, there is no major Python package for empirical attainment surface. We, therefore, develop a Python package for this purpose and describe the usage. The package is available at https://github.com/nabenabe0928/empirical-attainment-func.

翻译：超参数优化对于实现深度学习的高性能至关重要。除了性能之外，由于实际原因，通常还需要优化其他指标，例如推理时间或内存需求。这推动了多目标优化（MOO）方面的研究。然而，MOO方法的帕累托前沿在展示时往往未考虑随机种子引起的变异性，这使得性能稳定性评估变得困难。尽管存在名为经验达到面的概念，能够实现多次运行中带有不确定性的可视化，但目前尚无用于经验达到面的主要Python包。因此，我们为此目的开发了一个Python包，并描述了其使用方法。该包可在https://github.com/nabenabe0928/empirical-attainment-func获取。