This paper considers an ML inspired approach to hypothesis testing known as classifier/classification-accuracy testing ($\mathsf{CAT}$). In $\mathsf{CAT}$, one first trains a classifier by feeding it labeled synthetic samples generated by the null and alternative distributions, which is then used to predict labels of the actual data samples. This method is widely used in practice when the null and alternative are only specified via simulators (as in many scientific experiments). We study goodness-of-fit, two-sample ($\mathsf{TS}$) and likelihood-free hypothesis testing ($\mathsf{LFHT}$), and show that $\mathsf{CAT}$ achieves (near-)minimax optimal sample complexity in both the dependence on the total-variation ($\mathsf{TV}$) separation $\epsilon$ and the probability of error $\delta$ in a variety of non-parametric settings, including discrete distributions, $d$-dimensional distributions with a smooth density, and the Gaussian sequence model. In particular, we close the high probability sample complexity of $\mathsf{LFHT}$ for each class. As another highlight, we recover the minimax optimal complexity of $\mathsf{TS}$ over discrete distributions, which was recently established by Diakonikolas et al. (2021). The corresponding $\mathsf{CAT}$ simply compares empirical frequencies in the first half of the data, and rejects the null when the classification accuracy on the second half is better than random.

翻译：本文探讨一种受机器学习启发的假设检验方法——分类器/分类精度检验（$\mathsf{CAT}$）。该方法首先利用零假设和备择假设分布生成的带标签合成样本训练分类器，随后用该分类器预测实际数据样本的标签。此方法在零假设和备择假设仅通过模拟器指定时被广泛采用（例如众多科学实验）。我们研究拟合优度检验、双样本检验（$\mathsf{TS}$）及无似然假设检验（$\mathsf{LFHT}$），并证明在多种非参数设定下——包括离散分布、具有光滑密度的$d$维分布以及高斯序列模型——$\mathsf{CAT}$在总变差（$\mathsf{TV}$）分离度$\epsilon$和误差概率$\delta$的依赖关系上均达到（近似）极小极大最优样本复杂度。特别地，我们针对每个类别闭合了$\mathsf{LFHT}$的高概率样本复杂度。另一重要成果是，我们恢复了离散分布上$\mathsf{TS}$的极小极大最优复杂度，该结果近期由Diakonikolas等人（2021）建立。相应的$\mathsf{CAT}$方法仅比较数据前半部分的经验频率，当后半部分的分类精度优于随机水平时拒绝零假设。