Series or orthogonal basis regression is one of the most popular non-parametric regression techniques in practice, obtained by regressing the response on features generated by evaluating the basis functions at observed covariate values. The most routinely used series estimator is based on ordinary least squares fitting, which is known to be minimax rate optimal in various settings, albeit under stringent restrictions on the basis functions and the distribution of covariates. In this work, inspired by the recently developed Forster-Warmuth (FW) learner, we propose an alternative series regression estimator that can attain the minimax estimation rate under strictly weaker conditions imposed on the basis functions and the joint law of covariates, than existing series estimators in the literature. Moreover, a key contribution of this work generalizes the FW-learner to a so-called counterfactual regression problem, in which the response variable of interest may not be directly observed (hence, the name ``counterfactual'') on all sampled units, and therefore needs to be inferred in order to identify and estimate the regression in view from the observed data. Although counterfactual regression is not entirely a new area of inquiry, we propose the first-ever systematic study of this challenging problem from a unified pseudo-outcome perspective. In fact, we provide what appears to be the first generic and constructive approach for generating the pseudo-outcome (to substitute for the unobserved response) which leads to the estimation of the counterfactual regression curve of interest with small bias, namely bias of second order. Several applications are used to illustrate the resulting FW-learner including many nonparametric regression problems in missing data and causal inference literature, for which we establish high-level conditions for minimax rate optimality of the proposed FW-learner.

翻译：级数或正交基回归是实践中应用最广泛的非参数回归技术之一，其通过将响应变量对由基函数在观测协变量值上求值生成的特征进行回归而实现。最常规的级数估计量基于普通最小二乘拟合，虽在多种设定下具有极小化极大速率最优性，但需对基函数和协变量分布施加严格约束。受近期提出的Forster-Warmuth（FW）学习器启发，本文提出一种替代性级数回归估计量，该估计量能在对基函数与协变量联合分布施加比现有级数估计量更宽松的条件下，达到极小化极大估计速率。此外，本文的关键贡献在于将FW学习器推广至所谓的反事实回归问题——其中感兴趣的响应变量可能无法在全部采样单元上直接观测（故得名“反事实”），因此需通过推断从观测数据中识别并估计目标回归函数。尽管反事实回归并非全新研究领域，本文首次从统一的伪结果视角对此挑战性问题展开系统性研究。实际上，我们提出了首个通用且具有构造性的伪结果生成方法（用于替代未观测响应），该方法能以二阶小偏差估计目标反事实回归曲线。通过缺失数据与因果推断文献中的多个非参数回归问题实例，我们展示了所提出FW学习器的应用效果，并建立了该学习器达到极小化极大速率最优性的高层次条件。