Renewed interest in the relationship between artificial and biological neural networks motivates the study of gradient-free methods. Considering the linear regression model with random design, we theoretically analyze in this work the biologically motivated (weight-perturbed) forward gradient scheme that is based on random linear combination of the gradient. If d denotes the number of parameters and k the number of samples, we prove that the mean squared error of this method converges for $k\gtrsim d^2\log(d)$ with rate $d^2\log(d)/k.$ Compared to the dimension dependence d for stochastic gradient descent, an additional factor $d\log(d)$ occurs.

翻译：对人工神经网络与生物神经网络之间关系的新兴趣推动了无梯度方法的研究。本文考虑具有随机设计的线性回归模型，从理论上分析了基于梯度随机线性组合的（权重扰动）前向梯度方案（一种受生物学启发的方法）。若记d为参数数量，k为样本数量，我们证明该方法的均方误差在$k\gtrsim d^2\log(d)$条件下以$d^2\log(d)/k$的速率收敛。与随机梯度下降的维度依赖性d相比，该方法额外引入了因子$d\log(d)$。