Hyper-parameter optimization is the black art of tuning a learner's control parameters. In software analytics, a repeated result is that such tuning can result in dramatic performance improvements. Despite this, hyper-parameter optimization is often applied rarely or poorly in software analytics--perhaps due to the CPU cost of exploring all those parameter options can be prohibitive. We theorize that learners generalize better when the loss landscape is ``smooth''. This theory is useful since the influence on ``smoothness'' of different hyper-parameter choices can be tested very quickly (e.g. for a deep learner, after just one epoch). To test this theory, this paper implements and tests SMOOTHIE, a novel hyper-parameter optimizer that guides its optimizations via considerations of ``smothness''. The experiments of this paper test SMOOTHIE on numerous SE tasks including (a) GitHub issue lifetime prediction; (b) detecting false alarms in static code warnings; (c) defect prediction, and (d) a set of standard ML datasets. In all these experiments, SMOOTHIE out-performed state-of-the-art optimizers. Better yet, SMOOTHIE ran 300% faster than the prior state-of-the art. We hence conclude that this theory (that hyper-parameter optimization is best viewed as a ``smoothing'' function for the decision landscape), is both theoretically interesting and practically very useful. To support open science and other researchers working in this area, all our scripts and datasets are available on-line at https://github.com/yrahul3910/smoothness-hpo/.

翻译：超参数优化是微调学习器控制参数的“黑魔法”。在软件分析中，反复出现的结果表明，这种调优能够带来显著的性能提升。尽管如此，超参数优化在软件分析中往往很少被应用或应用效果不佳——这或许是因为探索所有参数选项的CPU成本可能高得令人望而却步。我们理论化认为，当损失景观“平滑”时，学习器的泛化能力更强。这一理论非常实用，因为不同超参数选择对“平滑度”的影响可以非常快速地测试（例如，对于深度学习器，仅需一个训练周期）。为验证该理论，本文实现并测试了SMOOTHIE——一种新颖的超参数优化器，它通过“平滑度”考量来引导优化过程。本文的实验在多个软件工程任务上测试了SMOOTHIE，包括：(a) GitHub问题生命周期预测；(b) 静态代码警告中的误报检测；(c) 缺陷预测；以及(d) 一组标准机器学习数据集。在所有实验中，SMOOTHIE均优于当前最先进的优化器。更值得一提的是，SMOOTHIE的运行速度比先前最先进方法快300%。因此，我们得出结论：这一理论（即超参数优化应被视为决策景观的“平滑”函数）既具有理论趣味性，又具有极高的实用价值。为支持开放科学及该领域其他研究人员，我们所有的脚本和数据集均可在https://github.com/yrahul3910/smoothness-hpo/ 在线获取。