The standard paired-sample testing approach in the multidimensional setting applies multiple univariate tests on the individual features, followed by p-value adjustments. Such an approach suffers when the data carry numerous features. A number of studies have shown that classification accuracy can be seen as a proxy for two-sample testing. However, neither theoretical foundations nor practical recipes have been proposed so far on how this strategy could be extended to multidimensional paired-sample testing. In this work, we put forward the idea that scoring functions can be produced by the decision rules defined by the perpendicular bisecting hyperplanes of the line segments connecting each pair of instances. Then, the optimal scoring function can be obtained by the pseudomedian of those rules, which we estimate by extending naturally the Hodges-Lehmann estimator. We accordingly propose a framework of a two-step testing procedure. First, we estimate the bisecting hyperplanes for each pair of instances and an aggregated rule derived through the Hodges-Lehmann estimator. The paired samples are scored by this aggregated rule to produce a unidimensional representation. Second, we perform a Wilcoxon signed-rank test on the obtained representation. Our experiments indicate that our approach has substantial performance gains in testing accuracy compared to the traditional multivariate and multiple testing, while at the same time estimates each feature's contribution to the final result.
翻译:标准的多维配对样本检验方法对各个特征分别进行单变量检验,然后进行p值校正。当数据包含大量特征时,这种方法效果不佳。多项研究表明,分类准确率可作为双样本检验的替代指标。然而,目前尚未有研究提出如何将此策略扩展到多维配对样本检验的理论基础或实践方法。本研究提出,可由连接每对实例的线段的中垂超平面所定义的决策规则生成评分函数。通过自然推广Hodges-Lehmann估计量,可获得这些规则的伪中位数作为最优评分函数。据此,我们提出一个两步检验框架:首先,估计每对实例的中垂超平面,并通过Hodges-Lehmann估计量推导聚合规则;配对样本经此聚合规则评分后生成一维表征。其次,对所得表征执行Wilcoxon符号秩检验。实验表明,与传统多变量检验及多重比较方法相比,本方法在检验准确率上具有显著提升,同时能估计每个特征对最终结果的贡献度。