Optimization in machine learning typically deals with the minimization of empirical objectives defined by training data. However, the ultimate goal of learning is to minimize the error on future data (test error), for which the training data provides only partial information. In this view, the optimization problems that are practically feasible are based on inexact quantities that are stochastic in nature. In this paper, we show how probabilistic results, specifically gradient concentration, can be combined with results from inexact optimization to derive sharp test error guarantees. By considering unconstrained objectives we highlight the implicit regularization properties of optimization for learning.

翻译：优化机器学习通常涉及最大限度地减少培训数据界定的经验性目标,然而,学习的最终目标是尽量减少未来数据的错误(测试错误),而培训数据只提供部分信息。在这种观点中,实际可行的优化问题是基于不精确的数量,而数量是随机的。在本文中,我们表明概率性结果,特别是梯度浓度,如何与不精确优化的结果相结合,以获得精确的测试错误保证。通过考虑不受限制的目标,我们强调优化学习的内在正规化特性。