We investigate a generalized framework for estimating latent low-rank tensors in an online setting, encompassing both linear and generalized linear models. This framework offers a flexible approach for handling continuous or categorical variables. Additionally, we investigate two specific applications: online tensor completion and online binary tensor learning. To address these challenges, we propose the online Riemannian gradient descent algorithm, which demonstrates linear convergence and the ability to recover the low-rank component under appropriate conditions in all applications. Furthermore, we establish a precise entry-wise error bound for online tensor completion. Notably, our work represents the first attempt to incorporate noise in the online low-rank tensor recovery task. Intriguingly, we observe a surprising trade-off between computational and statistical aspects in the presence of noise. Increasing the step size accelerates convergence but leads to higher statistical error, whereas a smaller step size yields a statistically optimal estimator at the expense of slower convergence. Moreover, we conduct regret analysis for online tensor regression. Under the fixed step size regime, a fascinating trilemma concerning the convergence rate, statistical error rate, and regret is observed. With an optimal choice of step size we achieve an optimal regret of $O(\sqrt{T})$. Furthermore, we extend our analysis to the adaptive setting where the horizon T is unknown. In this case, we demonstrate that by employing different step sizes, we can attain a statistically optimal error rate along with a regret of $O(\log T)$. To validate our theoretical claims, we provide numerical results that corroborate our findings and support our assertions.

翻译：我们研究了一个在在线设置中估计潜在低秩张量的广义框架，该框架涵盖线性模型和广义线性模型，为处理连续或分类变量提供了灵活的方法。此外，我们探讨了两个具体应用：在线张量补全和在线二元张量学习。为了解决这些挑战，我们提出了在线黎曼梯度下降算法，该算法在所有应用中均表现出线性收敛性，并在适当条件下能够恢复低秩分量。进一步地，我们为在线张量补全建立了精确的逐元素误差界。值得注意的是，我们的工作是首次将噪声纳入在线低秩张量恢复任务中的尝试。有趣的是，我们观察到在存在噪声的情况下，计算与统计方面存在一种令人惊讶的权衡：增大步长可加速收敛，但会导致更高的统计误差；而较小的步长能以较慢的收敛速度换取统计最优的估计量。此外，我们对在线张量回归进行了遗憾分析。在固定步长设置下，我们观察到一个关于收敛速率、统计误差率和遗憾的引人入胜的三难困境。通过选择最优步长，我们实现了$O(\sqrt{T})$的最优遗憾。进一步地，我们将分析扩展到时域T未知的自适应设置。在这种情况下，我们证明通过采用不同的步长，可以达到统计最优的误差率，同时实现$O(\log T)$的遗憾。为了验证我们的理论主张，我们提供了数值结果来佐证我们的发现并支持我们的论断。