For decades, best subset selection (BSS) has eluded statisticians mainly due to its computational bottleneck. However, until recently, modern computational breakthroughs have rekindled theoretical interest in BSS and have led to new findings. Recently, \cite{guo2020best} showed that the model selection performance of BSS is governed by a margin quantity that is robust to the design dependence, unlike modern methods such as LASSO, SCAD, MCP, etc. Motivated by their theoretical results, in this paper, we also study the variable selection properties of best subset selection for high-dimensional sparse linear regression setup. We show that apart from the identifiability margin, the following two complexity measures play a fundamental role in characterizing the margin condition for model consistency: (a) complexity of \emph{residualized features}, (b) complexity of \emph{spurious projections}. In particular, we establish a simple margin condition that depends only on the identifiability margin and the dominating one of the two complexity measures. Furthermore, we show that a margin condition depending on similar margin quantity and complexity measures is also necessary for model consistency of BSS. For a broader understanding, we also consider some simple illustrative examples to demonstrate the variation in the complexity measures that refines our theoretical understanding of the model selection performance of BSS under different correlation structures.

翻译：几十年来，最佳子集选择（BSS）因其计算瓶颈而一直困扰统计学家。然而，直到最近，现代计算方面的突破重新激发了人们对BSS的理论兴趣，并带来了新的发现。近期，\cite{guo2020best} 的研究表明，与LASSO、SCAD、MCP等现代方法不同，BSS的模型选择性能受一种对设计依赖性稳健的边际量支配。受其理论结果的启发，本文进一步研究了高维稀疏线性回归设定下最佳子集选择的变量选择特性。我们证明，除可辨识边际外，以下两种复杂度度量在刻画模型一致性的边际条件中起着基础性作用：(a) 残差化特征的复杂度；(b) 虚假投影的复杂度。特别地，我们建立了一个仅依赖于可辨识边际及这两种复杂度度量中主导项的简单边际条件。此外，我们证明依赖于类似边际量和复杂度度量的边际条件对于BSS的模型一致性也是必要的。为加深理解，我们还通过一些简单示例展示复杂度度量的变化，从而细化对不同相关结构下BSS模型选择性能的理论认识。