Optimization was recently shown to control the inductive bias in a learning process, a property referred to as implicit, or iterative regularization. The estimator obtained iteratively minimizing the training error can generalise well with no need of further penalties or constraints. In this paper, we investigate this phenomenon in the context of linear models with smooth loss functions. In particular, we investigate and propose a proof technique combining ideas from inexact optimization and probability theory, specifically gradient concentration. The proof is easy to follow and allows to obtain sharp learning bounds. More generally, it highlights a way to develop optimization results into learning guarantees.

翻译：最近,优化被证明控制了学习过程中的诱导偏差,即所谓的隐含或迭代规范化的财产。通过迭代最小化培训错误获得的估测者可以很好地概括培训错误,而不需要进一步的处罚或制约。在本文件中,我们从具有平稳损失功能的线性模型的角度来调查这一现象。特别是,我们调查并提出一种证据技术,将不精确优化和概率理论,特别是梯度集中的概念结合起来。证据很容易得到,并能够获得敏锐的学习界限。更一般地说,它突出了一种将优化结果发展为学习保障的方法。