Endogeneity, i.e. the dependence between noise and covariates, is a common phenomenon in real data due to omitted variables, strategic behaviours, measurement errors etc. In contrast, the existing analyses of stochastic online linear regression with unbounded noise and linear bandits depend heavily on exogeneity, i.e. the independence between noise and covariates. Motivated by this gap, we study the over-and just-identified Instrumental Variable (IV) regression for stochastic online learning. IV regression and the Two-Stage Least Squares approach to it are widely deployed in economics and causal inference to identify the underlying model from an endogenous dataset. Thus, we propose to use an online variant of Two-Stage Least Squares approach, namely O2SLS, to tackle endogeneity in stochastic online learning. Our analysis shows that O2SLS achieves $\mathcal{O}\left(d_x d_z \log ^2 T\right)$ identification and $\tilde{\mathcal{O}}\left(\gamma \sqrt{d_x T}\right)$ oracle regret after $T$ interactions, where $d_x$ and $d_z$ are the dimensions of covariates and IVs, and $\gamma$ is the bias due to endogeneity. For $\gamma=0$, i.e. under exogeneity, O2SLS achieves $\mathcal{O}\left(d_x^2 \log ^2 T\right)$ oracle regret, which is of the same order as that of the stochastic online ridge. Then, we leverage O2SLS as an oracle to design OFUL-IV, a stochastic linear bandit algorithm that can tackle endogeneity and achieves $\widetilde{\mathcal{O}}\left(\sqrt{d_x d_z T}\right)$ regret. For different datasets with endogeneity, we experimentally show efficiencies of O2SLS and OFUL-IV in terms of regrets.

翻译：内生性，即噪声与协变量之间的相关性，是真实数据中因遗漏变量、策略行为、测量误差等而常见的现象。然而，现有针对无界噪声随机在线线性回归和线性强盗的分析严重依赖于外生性，即噪声与协变量的独立性。受此差距启发，我们研究了用于随机在线学习的过度识别和恰好识别工具变量回归。工具变量回归及其两阶段最小二乘法广泛应用于经济学和因果推断中，用于从内生数据集中识别潜在模型。因此，我们提出使用两阶段最小二乘法的在线变体，即O2SLS，来处理随机在线学习中的内生性。我们的分析表明，在T次交互后，O2SLS实现了$\mathcal{O}\left(d_x d_z \log^2 T\right)$的识别误差和$\tilde{\mathcal{O}}\left(\gamma \sqrt{d_x T}\right)$的预言机遗憾，其中$d_x$和$d_z$分别是协变量和工具变量的维度，$\gamma$是由内生性导致的偏差。当$\gamma=0$，即在外生性条件下，O2SLS实现了$\mathcal{O}\left(d_x^2 \log^2 T\right)$的预言机遗憾，这与随机在线岭回归的量级相同。然后，我们利用O2SLS作为预言机，设计了OFUL-IV——一种能够处理内生性的随机线性强盗算法，并实现了$\widetilde{\mathcal{O}}\left(\sqrt{d_x d_z T}\right)$的遗憾。针对具有内生性的不同数据集，我们通过实验展示了O2SLS和OFUL-IV在遗憾方面的效率。