Renewed interest in the relationship between artificial and biological neural networks motivates the study of gradient-free methods. Considering the linear regression model with random design, we theoretically analyze in this work the biologically motivated (weight-perturbed) forward gradient scheme that is based on random linear combination of the gradient. If d denotes the number of parameters and k the number of samples, we prove that the mean squared error of this method converges for $k\gtrsim d^2\log(d)$ with rate $d^2\log(d)/k.$ Compared to the dimension dependence d for stochastic gradient descent, an additional factor $d\log(d)$ occurs.

翻译：人工神经网络与生物神经网络之间关系的新兴研究兴趣推动了无梯度方法的研究。考虑随机设计的线性回归模型，本文从理论上分析了基于梯度随机线性组合的生物启发式（权重扰动）前向梯度方案。若以d表示参数数量，k表示样本数量，我们证明当$k\gtrsim d^2\log(d)$时，该方法的均方误差以$d^2\log(d)/k$的速率收敛。与随机梯度下降的维度依赖d相比，该方法额外产生了$d\log(d)$的因子。