Two-sample testing tests whether the distributions generating two samples are identical. We pose the two-sample testing problem in a new scenario where the sample measurements (or sample features) are inexpensive to access, but their group memberships (or labels) are costly. We devise the first \emph{active sequential two-sample testing framework} that not only sequentially but also \emph{actively queries} sample labels to address the problem. Our test statistic is a likelihood ratio where one likelihood is found by maximization over all class priors, and the other is given by a classification model. The classification model is adaptively updated and then used to guide an active query scheme called bimodal query to label sample features in the regions with high dependency between the feature variables and the label variables. The theoretical contributions in the paper include proof that our framework produces an \emph{anytime-valid} $p$-value; and, under reachable conditions and a mild assumption, the framework asymptotically generates a minimum normalized log-likelihood ratio statistic that a passive query scheme can only achieve when the feature variable and the label variable have the highest dependence. Lastly, we provide a \emph{query-switching (QS)} algorithm to decide when to switch from passive query to active query and adapt bimodal query to increase the testing power of our test. Extensive experiments justify our theoretical contributions and the effectiveness of QS.

翻译：双样本检验用于检验生成两个样本的分布是否相同。我们提出了一种新的双样本检验场景：样本测量值（或样本特征）易于获取，但其组别归属（或标签）成本高昂。我们首创了**主动序贯双样本检验框架**，该框架不仅序贯地、还**主动地查询**样本标签以解决该问题。我们的检验统计量是似然比，其中一个似然度通过最大化所有类别先验获得，另一个则由分类模型给出。分类模型被自适应更新，并用于指导一种称为双模态查询的主动查询策略，以标记特征变量与标签变量之间高度依赖区域内的样本特征。本文的理论贡献包括：证明我们的框架能产生**任意有效**的$p$值；在可达条件与一个温和假设下，该框架渐近地生成最小归一化对数似然比统计量——而被动查询策略仅在特征变量与标签变量具有最高依赖关系时才能达到该统计量。最后，我们提出了**查询切换（QS）**算法，用于决定何时从被动查询切换至主动查询，并自适应地调整双模态查询以提升检验功效。大量实验验证了我们的理论贡献及QS算法的有效性。