As a fundamental concept in information theory, mutual information ($MI$) has been commonly applied to quantify association between random vectors. Most existing nonparametric estimators of $MI$ have unstable statistical performance since they involve parameter tuning. We develop a consistent and powerful estimator, called fastMI, that does not incur any parameter tuning. Based on a copula formulation, fastMI estimates $MI$ by leveraging Fast Fourier transform-based estimation of the underlying density. Extensive simulation studies reveal that fastMI outperforms state-of-the-art estimators with improved estimation accuracy and reduced run time for large data sets. fastMI provides a powerful test for independence that exhibits satisfactory type I error control. Anticipating that it will be a powerful tool in estimating mutual information in a broad range of data, we develop an R package fastMI for broader dissemination.
翻译:作为信息论中的基本概念,互信息($MI$)已被广泛应用于量化随机向量间的关联性。现有大多数$MI$的非参数估计器因需参数调优而导致统计性能不稳定。我们开发了一种无需参数调优的一致且高效的估计器fastMI。该估计器基于Copula框架,通过利用快速傅里叶变换估计潜在密度来实现$MI$的估计。大量仿真研究表明,fastMI在提升估计精度的同时,针对大规模数据集显著降低了运行时间,性能优于现有最优估计器。fastMI提供了具有满意第一类错误控制能力的独立性检验。预期其将成为广泛数据类型中互信息估计的有力工具,为此我们开发了R语言包fastMI以促进推广应用。