We investigate the statistical behavior of gradient descent iterates with dropout in the linear regression model. In particular, non-asymptotic bounds for the convergence of expectations and covariance matrices of the iterates are derived. The results shed more light on the widely cited connection between dropout and l2-regularization in the linear model. We indicate a more subtle relationship, owing to interactions between the gradient descent dynamics and the additional randomness induced by dropout. Further, we study a simplified variant of dropout which does not have a regularizing effect and converges to the least squares estimator

翻译：我们研究了线性回归模型中含dropout的梯度下降迭代的统计行为。具体地，推导了迭代期望与协方差矩阵收敛性的非渐近界。这些结果进一步揭示了线性模型中dropout与l2正则化之间广泛引用的关联。我们指出，由于梯度下降动态与dropout引入的额外随机性之间的相互作用，两者之间存在更为微妙的关系。此外，我们研究了一种简化版dropout，该变体不具有正则化效应，并收敛于最小二乘估计量。