We propose a new randomized method for solving systems of nonlinear equations, which can find sparse solutions or solutions under certain simple constraints. The scheme only takes gradients of component functions and uses Bregman projections onto the solution space of a Newton equation. In the special case of euclidean projections, the method is known as nonlinear Kaczmarz method. Furthermore, if the component functions are nonnegative, we are in the setting of optimization under the interpolation assumption and the method reduces to SGD with the recently proposed stochastic Polyak step size. For general Bregman projections, our method is a stochastic mirror descent with a novel adaptive step size. We prove that in the convex setting each iteration of our method results in a smaller Bregman distance to exact solutions as compared to the standard Polyak step. Our generalization to Bregman projections comes with the price that a convex one-dimensional optimization problem needs to be solved in each iteration. This can typically be done with globalized Newton iterations. Convergence is proved in two classical settings of nonlinearity: for convex nonnegative functions and locally for functions which fulfill the tangential cone condition. Finally, we show examples in which the proposed method outperforms similar methods with the same memory requirements.

翻译：我们提出一种新型随机化方法用于求解非线性方程组，该方法能够找到稀疏解或满足特定简单约束条件的解。该方案仅需利用分量函数的梯度，并通过Bregman投影将解投影至牛顿方程的解空间。在欧几里得投影的特殊情形下，该方法即为经典的非线性Kaczmarz方法。进一步地，当分量函数非负时，该方法可归入插值假设下的优化框架，并等价于采用近期提出的随机Polyak步长的随机梯度下降法。对于一般Bregman投影，我们的方法是一种具有新型自适应步长的随机镜像下降算法。我们证明在凸性条件下，与标准Polyak步长相比，该方法每次迭代能获得更小的Bregman距离精确解。将方法推广至Bregman投影的代价是每个迭代需求解一个凸一维优化问题，该问题通常可通过全局化的牛顿迭代实现。我们分别在两类经典非线性场景下证明了收敛性：一类是针对凸非负函数，另一类是在局部满足切锥条件的函数。最后通过算例展示，在相同内存需求条件下，所提方法性能优于同类算法。