We study the problem of online generalized linear regression in the stochastic setting, where the label is generated from a generalized linear model with possibly unbounded additive noise. We provide a sharp analysis of the classical follow-the-regularized-leader (FTRL) algorithm to cope with the label noise. More specifically, for $\sigma$-sub-Gaussian label noise, our analysis provides a regret upper bound of $O(\sigma^2 d \log T) + o(\log T)$, where $d$ is the dimension of the input vector, $T$ is the total number of rounds. We also prove a $\Omega(\sigma^2d\log(T/d))$ lower bound for stochastic online linear regression, which indicates that our upper bound is nearly optimal. In addition, we extend our analysis to a more refined Bernstein noise condition. As an application, we study generalized linear bandits with heteroscedastic noise and propose an algorithm based on FTRL to achieve the first variance-aware regret bound.

翻译：我们研究随机设定下的在线广义线性回归问题，其中标签由广义线性模型生成，可能伴随无界加性噪声。针对经典的前瞻正则化领导者（FTRL）算法应对标签噪声的情形，我们给出精确分析。具体而言，对于σ-次高斯型标签噪声，我们的分析得到遗憾上界为O(σ²d log T) + o(log T)，其中d为输入向量维度，T为总轮数。同时证明随机在线线性回归的Ω(σ²d log(T/d))下界，表明我们的上界接近最优。此外，我们将分析扩展到更精细的伯恩斯坦噪声条件。作为应用，我们研究含异方差噪声的广义线性赌博机，并提出基于FTRL的算法，首次实现方差感知的遗憾界。