Consider the problem of binary hypothesis testing. Given $Z$ coming from either $\mathbb P^{\otimes m}$ or $\mathbb Q^{\otimes m}$, to decide between the two with small probability of error it is sufficient and in most cases necessary to have $m \asymp 1/\epsilon^2$, where $\epsilon$ measures the separation between $\mathbb P$ and $\mathbb Q$ in total variation ($\mathsf{TV}$). Achieving this, however, requires complete knowledge of the distributions and can be done, for example, using the Neyman-Pearson test. In this paper we consider a variation of the problem, which we call likelihood-free (or simulation-based) hypothesis testing, where access to $\mathbb P$ and $\mathbb Q$ is given through $n$ iid observations from each. In the case when $\mathbb P,\mathbb Q$ are assumed to belong to a non-parametric family $\mathcal P$, we demonstrate the existence of a fundamental trade-off between $n$ and $m$ given by $nm \asymp n^2_\mathsf{GoF}(\epsilon,\cal P)$, where $n_\mathsf{GoF}$ is the minimax sample complexity of testing between the hypotheses $H_0: \mathbb P= \mathbb Q$ vs $H_1: \mathsf{TV}(\mathbb P,\mathbb Q) \ge \epsilon$. We show this for three families of distributions: $\beta$-smooth densities supported on $[0,1]^d$, the Gaussian sequence model over a Sobolev ellipsoid, and the collection of distributions on alphabet $[k]=\{1,2,\dots,k\}$ with pmfs bounded by $c/k$ for fixed $c$. For the larger family of all distributions on $[k]$ we obtain a more complicated trade-off that exhibits a phase-transition. The test that we propose, based on the $L^2$-distance statistic of Ingster, simultaneously achieves all points on the trade-off curve for the regular classes. This demonstrates the possibility of testing without fully estimating the distributions, provided $m\gg1/\epsilon^2$.

翻译：考虑二元假设检验问题。给定来自 $\mathbb P^{\otimes m}$ 或 $\mathbb Q^{\otimes m}$ 的 $Z$，要在错误概率较小的前提下在两者间进行判定，充分且大多数情况下必要的条件是 $m \asymp 1/\epsilon^2$，其中 $\epsilon$ 衡量 $\mathbb P$ 与 $\mathbb Q$ 在总变差（$\mathsf{TV}$）下的分离程度。然而，实现这一条件需要完全掌握分布信息，例如可通过奈曼-皮尔逊检验完成。本文考虑该问题的一种变体，称为免似然（或基于模拟的）假设检验，其中每个分布 $\mathbb P$ 和 $\mathbb Q$ 的访问通过 $n$ 个独立同分布观测实现。当 $\mathbb P,\mathbb Q$ 属于非参数族 $\mathcal P$ 时，我们证明存在 $n$ 与 $m$ 之间的基本权衡关系：$nm \asymp n^2_\mathsf{GoF}(\epsilon,\cal P)$，其中 $n_\mathsf{GoF}$ 是检验假设 $H_0: \mathbb P= \mathbb Q$ 与 $H_1: \mathsf{TV}(\mathbb P,\mathbb Q) \ge \epsilon$ 的极小极大样本复杂度。我们针对三类分布族证明了该结论：支撑于 $[0,1]^d$ 的 $\beta$ 光滑密度函数、索伯列夫椭球上的高斯序列模型、以及字母表 $[k]=\{1,2,\dots,k\}$ 上概率质量函数受限于 $c/k$（$c$ 为固定常数）的分布集合。对于 $[k]$ 上所有分布构成的更大族，我们得到更复杂的权衡关系，该关系呈现相变现象。我们基于 Ingster 的 $L^2$ 距离统计量提出的检验方法，能同时实现正则类权衡曲线上的所有点。这证明了在满足 $m\gg1/\epsilon^2$ 的条件下，无需完全估计分布即可完成检验的可能性。