Study samples often differ from the target populations of inference and policy decisions in non-random ways. Researchers typically believe that such departures from random sampling -- due to changes in the population over time and space, or difficulties in sampling truly randomly -- are small, and their corresponding impact on the inference should be small as well. We might therefore be concerned if the conclusions of our studies are excessively sensitive to a very small proportion of our sample data. We propose a method to assess the sensitivity of applied econometric conclusions to the removal of a small fraction of the sample. Manually checking the influence of all possible small subsets is computationally infeasible, so we use an approximation to find the most influential subset. Our metric, the "Approximate Maximum Influence Perturbation," is based on the classical influence function, and is automatically computable for common methods including (but not limited to) OLS, IV, MLE, GMM, and variational Bayes. We provide finite-sample error bounds on approximation performance. At minimal extra cost, we provide an exact finite-sample lower bound on sensitivity. We find that sensitivity is driven by a signal-to-noise ratio in the inference problem, is not reflected in standard errors, does not disappear asymptotically, and is not due to misspecification. While some empirical applications are robust, results of several influential economics papers can be overturned by removing less than 1% of the sample.

翻译：研究样本通常以非随机方式与推断和政策决策的目标总体存在差异。研究者通常认为，这种偏离随机抽样的现象——源于总体随时间与空间的变化，或真正随机抽样的困难——是微小的，其对推断的影响也应当是微小的。因此，若研究结论对样本数据中极小比例的部分过度敏感，我们便应予以关注。本文提出一种评估应用计量经济学结论对少量样本剔除敏感性的方法。手动检验所有可能小子集的影响在计算上不可行，因此我们采用近似方法寻找影响力最大的子集。我们的度量指标——“近似最大影响扰动”——基于经典影响函数，可自动计算于常见方法（包括但不限于OLS、IV、MLE、GMM和变分贝叶斯）。我们提供了近似性能的有限样本误差界。以极小的额外成本，我们给出了敏感性的精确有限样本下界。研究发现：敏感性由推断问题中的信噪比驱动，不反映于标准误，不会渐近消失，且非模型误设所致。尽管部分实证应用具有稳健性，但若干具有影响力的经济学论文结论可通过剔除不足1%的样本而被推翻。