The contamination detection problem aims to determine whether a set of observations has been contaminated, i.e. whether it contains points drawn from a distribution different from the reference distribution. Here, we consider a supervised problem, where labeled samples drawn from both the reference distribution and the contamination distribution are available at training time. This problem is motivated by the detection of rare cells in flow cytometry. Compared to novelty detection problems or two-sample testing, where only samples from the reference distribution are available, the challenge lies in efficiently leveraging the observations from the contamination detection to design more powerful tests. In this article, we introduce a test for the supervised contamination detection problem. We provide non-asymptotic guarantees on its Type I error, and characterize its detection rate. The test relies on estimating reference and contamination densities using histograms, and its power depends strongly on the choice of the corresponding partition. We present an algorithm for judiciously choosing the partition that results in a powerful test. Simulations illustrate the good empirical performances of our partition selection algorithm and the efficiency of our test. Finally, we showcase our method and apply it to a real flow cytometry dataset.
翻译:污染检测问题旨在判断一组观测值是否受到污染,即其中是否包含来自与参考分布不同分布的点。本文考虑一个监督问题,在训练阶段可获得来自参考分布和污染分布的带标签样本。该问题源于流式细胞术中稀有细胞的检测。相较于仅依赖参考分布样本的新颖性检测或双样本检验问题,其挑战在于如何高效利用污染检测中的观测值来设计更具效力的检验方法。本文针对监督污染检测问题提出一种检验方法,并给出其第一类错误的非渐近保证,同时刻画了检测率。该检验通过直方图估计参考密度与污染密度,其检验功效高度依赖于对应划分的选择。我们提出一种算法来合理选择划分,从而获得高功效的检验。仿真实验展示了划分选择算法的良好实证性能及检验方法的有效性。最后,将所提方法应用于真实流式细胞术数据集进行验证。