We investigate a generalized framework for estimating latent low-rank tensors in an online setting, encompassing both linear and generalized linear models. This framework offers a flexible approach for handling continuous or categorical variables. Additionally, we investigate two specific applications: online tensor completion and online binary tensor learning. To address these challenges, we propose the online Riemannian gradient descent algorithm, which demonstrates linear convergence and the ability to recover the low-rank component under appropriate conditions in all applications. Furthermore, we establish a precise entry-wise error bound for online tensor completion. Notably, our work represents the first attempt to incorporate noise in the online low-rank tensor recovery task. Intriguingly, we observe a surprising trade-off between computational and statistical aspects in the presence of noise. Increasing the step size accelerates convergence but leads to higher statistical error, whereas a smaller step size yields a statistically optimal estimator at the expense of slower convergence. Moreover, we conduct regret analysis for online tensor regression. Under the fixed step size regime, a fascinating trilemma concerning the convergence rate, statistical error rate, and regret is observed. With an optimal choice of step size we achieve an optimal regret of $O(\sqrt{T})$. Furthermore, we extend our analysis to the adaptive setting where the horizon T is unknown. In this case, we demonstrate that by employing different step sizes, we can attain a statistically optimal error rate along with a regret of $O(\log T)$. To validate our theoretical claims, we provide numerical results that corroborate our findings and support our assertions.

翻译：我们研究了一种在在线环境中估计潜在低秩张量的广义框架，涵盖线性模型和广义线性模型。该框架为处理连续或分类变量提供了一种灵活的方法。此外，我们探讨了两个具体应用：在线张量补全和在线二值张量学习。为应对这些挑战，我们提出了在线黎曼梯度下降算法，该算法在所有应用场景中均表现出线性收敛性，并在适当条件下能够恢复低秩分量。进一步地，我们为在线张量补全建立了精确的逐元素误差界。值得注意的是，我们的工作是首次在在线低秩张量恢复任务中考虑噪声。有趣的是，我们在存在噪声的情况下观察到计算与统计之间的一种惊人权衡：增大步长可加速收敛，但会导致更高的统计误差；而减小步长虽能获得统计最优的估计量，但收敛速度较慢。此外，我们对在线张量回归进行了遗憾分析。在固定步长机制下，我们观察到关于收敛速率、统计误差率和遗憾的一个迷人三角困境。通过最优选择步长，我们实现了$O(\sqrt{T})$的最优遗憾。进一步地，我们将分析扩展到水平T未知的自适应场景。在此情况下，我们证明通过采用不同步长，可以在获得统计最优误差率的同时实现$O(\log T)$的遗憾。为验证我们的理论主张，我们提供了支持性数值结果。