This work establishes a theoretical foundation for joint feature selection with multivariate outcomes, positioning the permutation-based variable importance measure (PVIM) of multivariate random forests (MRF) as a principled tool for high-dimensional feature selection. We establish the first consistency guaranty for MRF, showing that it retains all truly influential features with probability tending to one as the sample size grows to infinity under mild regularity conditions. Incomplete U-statistics is employed to incorporate three layers of randomness: subsampling of subjects for training each tree, subsampling of features at each split, and permutation of each feature for the out-of-bag (OOB) samples. Unlike independence-based screening that evaluates each feature in isolation, PVIM is a joint screening approach that accounts for multicollinearity, nonlinear, high-order interactions, and subject heterogeneity via ensemble aggregation. Moreover, we demonstrate the practical utility of MRF through a genome-wide association study (GWAS) of human facial morphology (with 2,342 subjects and 453,273 SNPs), where MRF identifies several novel loci and interaction hubs that extend prior findings. Extensive simulations show that MRF accurately identifies truly influential signals while producing parsimonious feature sets with well controlled false selection rates, outperforming canonical correlation analysis (CCA) and several other independence multivariate screening approaches. In addition, we also propose a novel simulation framework, including image outcomes, that more closely mimic the intricate nature of real-world data and provide rigorous testbeds for machine learning research.

翻译：暂无翻译