A novel, stochastic approach to amputation, the process of introducing missing values to a complete dataset, is presented. It allows one to construct a wide variety of missingness patterns by only having to specify distributions of missingness indicators as opposed to specifying each missingness pattern manually. Missingness indicators are modeled in a principled way via copulas and Bernoulli margins, thus allowing one to incorporate dependence in missingness patterns. Besides more classical missingness mechanisms such as missing completely at random, missing at random, and missing not at random, the approach is able to model structured missingness such as block missingness and, via mixtures, monotone missingness, which are patterns of missing data frequently found in real-life datasets. Properties such as joint missingness probabilities or missingness correlation are derived mathematically. The flexibility of the approach in capturing different missingness patterns while only requiring to specify distributional assumptions on missingness indicators is demonstrated with mathematical examples and empirical illustrations in terms of a well-known example dataset of sufficiently small sample size that allows to identify each missing data point visually. Finally, an example application to multivariate financial time series is provided.
翻译:本文提出了一种新颖的随机截断方法,即向完整数据集中引入缺失值的过程。该方法仅需指定缺失指示变量的分布,而非逐一设定每种缺失模式,即可构建多种多样的缺失数据模式。基于copula函数和伯努利边际分布,缺失指示变量被以统一原理建模,从而能够刻画缺失模式间的相依性。除经典缺失机制(如完全随机缺失、随机缺失和非随机缺失)外,该方法还可建模结构化缺失模式(如区块缺失)及通过混合模型实现单调缺失——这些模式常见于真实数据集。本文从数学上推导了联合缺失概率、缺失相关性等性质。通过数学示例及基于经典小样本数据集的实证可视化(确保每个缺失数据点均可肉眼识别),论证了该方法仅需对缺失指示变量设定分布假设即可灵活捕捉不同缺失模式的特性。最后,以多元金融时间序列为例展示了应用场景。