Gradient descent on logistic regression has many fascinating properties not fully captured by classical analyses. For linearly separable data, the iterates converge in direction to the maximum-margin separator regardless of how large the step size is. For non-separable data, however, cycles may occur even for step sizes below the local stability threshold of $2/λ^*$, where $λ^*$ is the largest eigenvalue of the Hessian at the solution. We ask whether requiring all examples to have equal norm is a sufficient condition for global convergence below this threshold. We prove that this is indeed true in one dimension. In higher dimensions, even equal-norm datasets can produce cycles, although global convergence holds for the population loss when the examples are uniformly distributed on the sphere. Finally, in one dimension, we relax the equal-norm assumption to an annulus condition that ensures global convergence below the stability threshold, suggesting a path towards analogous sufficient conditions in higher dimensions.

翻译：暂无翻译