Endogeneity, i.e. the dependence of noise and covariates, is a common phenomenon in real data due to omitted variables, strategic behaviours, measurement errors etc. In contrast, the existing analyses of stochastic online linear regression with unbounded noise and linear bandits depend heavily on exogeneity, i.e. the independence of noise and covariates. Motivated by this gap, we study the over- and just-identified Instrumental Variable (IV) regression, specifically Two-Stage Least Squares, for stochastic online learning, and propose to use an online variant of Two-Stage Least Squares, namely O2SLS. We show that O2SLS achieves $\mathcal O(d_{x}d_{z}\log^2 T)$ identification and $\widetilde{\mathcal O}(\gamma \sqrt{d_{z} T})$ oracle regret after $T$ interactions, where $d_{x}$ and $d_{z}$ are the dimensions of covariates and IVs, and $\gamma$ is the bias due to endogeneity. For $\gamma=0$, i.e. under exogeneity, O2SLS exhibits $\mathcal O(d_{x}^2 \log^2 T)$ oracle regret, which is of the same order as that of the stochastic online ridge. Then, we leverage O2SLS as an oracle to design OFUL-IV, a stochastic linear bandit algorithm to tackle endogeneity. OFUL-IV yields $\widetilde{\mathcal O}(\sqrt{d_{x}d_{z}T})$ regret that matches the regret lower bound under exogeneity. For different datasets with endogeneity, we experimentally show efficiencies of O2SLS and OFUL-IV.

翻译：内生性，即噪声与协变量之间的依赖性，是实际数据中因遗漏变量、策略行为、测量误差等引发的常见现象。然而，现有针对含无界噪声的随机在线线性回归及线性赌博机的分析严重依赖于外生性，即噪声与协变量的独立性。受这一研究空白的驱动，我们研究了随机在线学习中过度识别与恰好识别情形下的工具变量回归（特别是两阶段最小二乘法），并提出其在线变体O2SLS。研究表明，经过T次交互后，O2SLS可实现$\mathcal O(d_{x}d_{z}\log^2 T)$的识别误差与$\widetilde{\mathcal O}(\gamma \sqrt{d_{z} T})$的预言机遗憾，其中$d_{x}$与$d_{z}$分别为协变量和工具变量的维度，$\gamma$为内生性导致的偏差。当$\gamma=0$（即外生性条件下），O2SLS的预言机遗憾为$\mathcal O(d_{x}^2 \log^2 T)$，与随机在线岭回归量级相同。进一步地，我们以O2SLS为预言机设计出OFUL-IV算法——一种应对内生性的随机线性赌博机算法。该算法可实现$\widetilde{\mathcal O}(\sqrt{d_{x}d_{z}T})$的遗憾，与外生性条件下的遗憾下界匹配。通过多组含内生性数据集的实验，我们验证了O2SLS与OFUL-IV的有效性。