In this paper we consider linearly constrained optimization problems and propose a loopless projection stochastic approximation (LPSA) algorithm. It performs the projection with probability $p_n$ at the $n$-th iteration to ensure feasibility. Considering a specific family of the probability $p_n$ and step size $\eta_n$, we analyze our algorithm from an asymptotic and continuous perspective. Using a novel jump diffusion approximation, we show that the trajectories connecting those properly rescaled last iterates weakly converge to the solution of specific stochastic differential equations (SDEs). By analyzing SDEs, we identify the asymptotic behaviors of LPSA for different choices of $(p_n, \eta_n)$. We find that the algorithm presents an intriguing asymptotic bias-variance trade-off and yields phase transition phenomenons, according to the relative magnitude of $p_n$ w.r.t. $\eta_n$. This finding provides insights on selecting appropriate ${(p_n, \eta_n)}_{n \geq 1}$ to minimize the projection cost. Additionally, we propose the Debiased LPSA (DLPSA) as a practical application of our jump diffusion approximation result. DLPSA is shown to effectively reduce projection complexity compared to vanilla LPSA.

翻译：本文考虑线性约束优化问题，提出了一种无环投影随机逼近算法。该算法在第n次迭代中以概率p_n执行投影以确保可行性。针对特定的概率序列p_n和步长η_n，我们从渐近与连续视角分析了算法性能。通过引入新颖的跳跃扩散逼近方法，我们证明了经过适当重标度的末次迭代轨迹弱收敛到特定随机微分方程的解。通过分析随机微分方程，我们识别了不同(p_n, η_n)选择下算法呈现的渐近行为。研究发现，根据p_n相对于η_n的幅度大小，算法呈现出有趣的渐近偏差-方差权衡并产生相变现象。该发现为选择最优项目代价的{(p_n, η_n)}_{n≥1}参数提供了理论依据。此外，基于跳跃扩散逼近结果，我们提出了去偏LPSA算法作为实际应用。理论证明，相较于原始LPSA，去偏LPSA能有效降低投影复杂度。