Stochastic coordinate descent algorithms are efficient methods in which each iterate is obtained by fixing most coordinates at their values from the current iteration, and approximately minimizing the objective with respect to the remaining coordinates. However, this approach is usually restricted to canonical basis vectors of $\mathbb{R}^d$. In this paper, we develop a new class of stochastic gradient descent algorithms with random search directions which uses the directional derivative of the gradient estimate following more general random vectors. We establish the almost sure convergence of these algorithms with decreasing step. We further investigate their central limit theorem and pay particular attention to analyze the impact of the search distributions on the asymptotic covariance matrix. We also provide non-asymptotic $\mathbb{L}^p$ rates of convergence.

翻译：随机坐标下降算法是一类高效的方法，其每次迭代通过将大部分坐标固定在当前迭代值，并对剩余坐标近似最小化目标函数来获得新迭代点。然而，这种方法通常局限于使用 $\mathbb{R}^d$ 的规范基向量。本文提出了一类新的具有随机搜索方向的随机梯度下降算法，该算法采用更一般的随机向量方向上的梯度估计方向导数。我们建立了这些算法在步长递减条件下的几乎必然收敛性。进一步研究了其中心极限定理，并特别分析了搜索分布对渐近协方差矩阵的影响。同时给出了非渐近的 $\mathbb{L}^p$ 收敛速率。