Series or orthogonal basis regression is one of the most popular non-parametric regression techniques in practice, obtained by regressing the response on features generated by evaluating the basis functions at observed covariate values. The most routinely used series estimator is based on ordinary least squares fitting, which is known to be minimax rate optimal in various settings, albeit under stringent restrictions on the basis functions and the distribution of covariates. In this work, inspired by the recently developed Forster-Warmuth (FW) learner, we propose an alternative series regression estimator that can attain the minimax estimation rate under strictly weaker conditions imposed on the basis functions and the joint law of covariates, than existing series estimators in the literature. Moreover, a key contribution of this work generalizes the FW-learner to a so-called counterfactual regression problem, in which the response variable of interest may not be directly observed (hence, the name ``counterfactual'') on all sampled units, and therefore needs to be inferred in order to identify and estimate the regression in view from the observed data. Although counterfactual regression is not entirely a new area of inquiry, we propose the first-ever systematic study of this challenging problem from a unified pseudo-outcome perspective. In fact, we provide what appears to be the first generic and constructive approach for generating the pseudo-outcome (to substitute for the unobserved response) which leads to the estimation of the counterfactual regression curve of interest with small bias, namely bias of second order. Several applications are used to illustrate the resulting FW-learner including many nonparametric regression problems in missing data and causal inference literature, for which we establish high-level conditions for minimax rate optimality of the proposed FW-learner.

翻译：级数或正交基回归是实践中应用最广泛的非参数回归技术之一，该方法通过将响应变量对由基函数在观测协变量值处求值生成的特征进行回归来实现。最常使用的级数估计量基于普通最小二乘拟合，已知其虽然在各种设定下能达到极小化最优率，但严格依赖于基函数和协变量分布的约束条件。本文受近期发展的福斯特-沃穆斯（FW）学习器启发，提出了一种替代性级数回归估计量，该估计量能在比现有文献中级数估计量更弱的基函数与协变量联合分布条件下达到极小化估计率。此外，本研究的重要贡献在于将FW学习器推广至所谓的反事实回归问题：其中感兴趣的响应变量可能无法在所有采样单元上直接观测（因此得名"反事实"），需通过推断才能从观测数据中识别并估计目标回归函数。尽管反事实回归并非全新研究领域，但我们首次从统一的伪结局视角对该挑战性问题进行了系统性研究。实际上，我们提出了首个普适且具建构性的伪结局生成方法（用于替代未观测响应），该方法的偏差仅为二阶小量，从而能以低偏差估计所关注的反事实回归曲线。本文通过多个应用实例展示所提出的FW学习器，涵盖缺失数据与因果推断文献中的多种非参数回归问题，并为所提出FW学习器的极小化最优率建立了高层条件。