Numerical data imputation algorithms replace missing values by estimates to leverage incomplete data sets. Current imputation methods seek to minimize the error between the unobserved ground truth and the imputed values. But this strategy can create artifacts leading to poor imputation in the presence of multimodal or complex distributions. To tackle this problem, we introduce the $k$NN$\times$KDE algorithm: a data imputation method combining nearest neighbor estimation ($k$NN) and density estimation with Gaussian kernels (KDE). We compare our method with previous data imputation methods using artificial and real-world data with different data missing scenarios and various data missing rates, and show that our method can cope with complex original data structure, yields lower data imputation errors, and provides probabilistic estimates with higher likelihood than current methods. We release the code in open-source for the community: https://github.com/DeltaFloflo/knnxkde
翻译:数值数据插补算法通过估计值替换缺失值,以利用不完整的数据集。当前的插补方法旨在最小化未观测的真实值与插补值之间的误差。然而,这种策略可能产生伪影,导致在面对多模态或复杂分布时插补效果不佳。为解决这一问题,我们提出$k$NN$\times$KDE算法:一种结合近邻估计($k$NN)与高斯核密度估计(KDE)的数据插补方法。我们通过人工和真实数据,在不同数据缺失场景及不同缺失率下,将我们的方法与现有数据插补方法进行比较,结果表明,我们的方法能够处理复杂的原始数据结构,降低数据插补误差,并提供比当前方法具有更高似然性的概率估计。我们以开源形式发布代码供社区使用:https://github.com/DeltaFloflo/knnxkde