Consider the problem of binary hypothesis testing. Given $Z$ coming from either $\mathbb P^{\otimes m}$ or $\mathbb Q^{\otimes m}$, to decide between the two with small probability of error it is sufficient and in most cases necessary to have $m \asymp 1/\epsilon^2$, where $\epsilon$ measures the separation between $\mathbb P$ and $\mathbb Q$ in total variation ($\mathsf{TV}$). Achieving this, however, requires complete knowledge of the distributions and can be done, for example, using the Neyman-Pearson test. In this paper we consider a variation of the problem, which we call likelihood-free (or simulation-based) hypothesis testing, where access to $\mathbb P$ and $\mathbb Q$ is given through $n$ iid observations from each. In the case when $\mathbb P,\mathbb Q$ are assumed to belong to a non-parametric family $\mathcal P$, we demonstrate the existence of a fundamental trade-off between $n$ and $m$ given by $nm \asymp n^2_\mathsf{GoF}(\epsilon,\cal P)$, where $n_\mathsf{GoF}$ is the minimax sample complexity of testing between the hypotheses $H_0: \mathbb P= \mathbb Q$ vs $H_1: \mathsf{TV}(\mathbb P,\mathbb Q) \ge \epsilon$. We show this for three families of distributions: $\beta$-smooth densities supported on $[0,1]^d$, the Gaussian sequence model over a Sobolev ellipsoid, and the collection of distributions on alphabet $[k]=\{1,2,\dots,k\}$ with pmfs bounded by $c/k$ for fixed $c$. For the larger family of all distributions on $[k]$ we obtain a more complicated trade-off that exhibits a phase-transition. The test that we propose, based on the $L^2$-distance statistic of Ingster, simultaneously achieves all points on the trade-off curve for the regular classes. This demonstrates the possibility of testing without fully estimating the distributions, provided $m\gg1/\epsilon^2$.

翻译：考虑二元假设检验问题。给定来自 $\mathbb P^{\otimes m}$ 或 $\mathbb Q^{\otimes m}$ 的 $Z$，要在两者之间以较小的错误概率做出判断，充分且多数情况下必要条件是 $m \asymp 1/\epsilon^2$，其中 $\epsilon$ 衡量 $\mathbb P$ 与 $\mathbb Q$ 在总变差（$\mathsf{TV}$）上的分离程度。然而，实现这一点需要完全知道分布，并且可以借助例如Neyman-Pearson检验完成。本文考虑该问题的一个变体，称为无似然（或基于模拟的）假设检验，其中通过每个分布的 $n$ 个独立同分布观测获取对 $\mathbb P$ 和 $\mathbb Q$ 的访问。当 $\mathbb P,\mathbb Q$ 属于非参数族 $\mathcal P$ 时，我们证明了 $n$ 与 $m$ 之间存在一个基本权衡关系：$nm \asymp n^2_\mathsf{GoF}(\epsilon,\cal P)$，其中 $n_\mathsf{GoF}$ 是检验假设 $H_0: \mathbb P= \mathbb Q$ 与 $H_1: \mathsf{TV}(\mathbb P,\mathbb Q) \ge \epsilon$ 的极小极大样本复杂度。我们针对三族分布展示了这一结果：支撑在 $[0,1]^d$ 上的 $\beta$ 光滑密度、Sobolev椭球上的高斯序列模型，以及字母表 $[k]=\{1,2,\dots,k\}$ 上概率质量函数有界于 $c/k$（$c$ 固定）的分布族。对于 $[k]$ 上所有分布构成的更大族，我们获得了更复杂的权衡关系，并呈现出相变现象。我们提出的基于Ingster $L^2$距离统计量的检验方法，能同时达到正则类权衡曲线上的所有点。这表明，只要 $m\gg1/\epsilon^2$，无需完全估计分布即可进行检验。