This paper provides a non-asymptotic analysis of linear stochastic approximation (LSA) algorithms with fixed stepsize. This family of methods arises in many machine learning tasks and is used to obtain approximate solutions of a linear system $\bar{A}\theta = \bar{b}$ for which $\bar{A}$ and $\bar{b}$ can only be accessed through random estimates $\{({\bf A}_n, {\bf b}_n): n \in \mathbb{N}^*\}$. Our analysis is based on new results regarding moments and high probability bounds for products of matrices which are shown to be tight. We derive high probability bounds on the performance of LSA under weaker conditions on the sequence $\{({\bf A}_n, {\bf b}_n): n \in \mathbb{N}^*\}$ than previous works. However, in contrast, we establish polynomial concentration bounds with order depending on the stepsize. We show that our conclusions cannot be improved without additional assumptions on the sequence of random matrices $\{{\bf A}_n: n \in \mathbb{N}^*\}$, and in particular that no Gaussian or exponential high probability bounds can hold. Finally, we pay a particular attention to establishing bounds with sharp order with respect to the number of iterations and the stepsize and whose leading terms contain the covariance matrices appearing in the central limit theorems.

翻译：本文以固定步骤化的线性近似( LSA) 算法提供非非不端分析 。 这一系列方法来自许多机器学习任务, 用于获取线性系统$\bar{ A ⁇ theta =\bar{b}$\b}$\bar{A}$和$\bar{b}$, 只能通过随机估算 ${( bff A}}) 和$\bar{bb} $( b ⁇ n) 来访问。 然而, 我们的分析基于关于显示为紧凑的矩阵产品时刻和高概率范围的新结果。 在 $\\\\ b} 和\ b} b} 的顺序下, 在较弱的条件下, $\ b ⁇, $\ 美元和 bar{ b\ b\ } 美元 的顺序上, 只能通过 随机估计来访问 $\\\ b} ( b\ b) 和 { b} $\ b\ b} $ 来获取 。 然而, 我们建立多边集中集中的界限, 取决于步骤的顺序 。 我们的结论不能改进 。 我们的结论在随机矩阵序列 $\\\\\\\\\\\\\\\\\\\\\\\\\\\\ rass rass rass rass rb rass rass rass rass rass rass rx rx rx 中, rx rx rx rx rx 中, 中, rx rx rx rx rx rx rx rx rx rx rx rx rx rx rx rx rx rx rx rx rx rx rx rx rx rx rx rx rx rx rx rx rx rx rx rx rx rx rx rx rx rx rx rx rx rx rx rx rx rx rx rx rx