Empirical risk minimization stands behind most optimization in supervised machine learning. Under this scheme, labeled data is used to approximate an expected cost (risk), and a learning algorithm updates model-defining parameters in search of an empirical risk minimizer, with the aim of thereby approximately minimizing expected cost. Parameter update is often done by some sort of gradient descent. In this paper, we introduce a learning algorithm to construct models for real analytic functions using neither gradient descent nor empirical risk minimization. Observing that such functions are defined by local information, we situate familiar Taylor approximation methods in the context of sampling data from a distribution, and prove a nonuniform learning result.
翻译:在监督式机器学习中,大多数优化方法都基于经验风险最小化。该方案利用标注数据来近似期望成本(风险),并通过学习算法更新模型参数以寻找经验风险极小化器,从而期望近似地最小化期望成本。参数更新通常采用某种形式的梯度下降法。本文提出了一种学习算法,该算法既不依赖梯度下降也不依赖经验风险最小化,而是构建用于实解析函数的模型。基于此类函数由局部信息定义这一观察,我们将经典的泰勒逼近方法置于从分布中采样数据的情境中,并证明了非一致学习结果。