With the modern software and online platforms to collect massive amount of data, there is an increasing demand of applying causal inference methods at large scale when randomized experimentation is not viable. Weighting methods that directly incorporate covariate balancing have recently gained popularity for estimating causal effects in observational studies. These methods reduce the manual efforts required by researchers to iterate between propensity score modeling and balance checking until a satisfied covariate balance result. However, conventional solvers for determining weights lack the scalability to apply such methods on large scale datasets in companies like Snap Inc. To address the limitations and improve computational efficiency, in this paper we present scalable algorithms, DistEB and DistMS, for two balancing approaches: entropy balancing and MicroSynth. The solvers have linear time complexity and can be conveniently implemented in distributed computing frameworks such as Spark, Hive, etc. We study the properties of balancing approaches at different scales up to 1 million treated units and 487 covariates. We find that with larger sample size, both bias and variance in the causal effect estimation are significantly reduced. The results emphasize the importance of applying balancing approaches on large scale datasets. We combine the balancing approach with a synthetic control framework and deploy an end-to-end system for causal impact estimation at Snap Inc.

翻译：随着现代软件和在线平台能够收集海量数据，在随机实验不可行的情况下，大规模应用因果推断方法的需求日益增长。近年来，直接纳入协变量平衡的加权方法在观测性研究的因果效应估计中备受青睐。这些方法减少了研究者手动迭代倾向得分建模与平衡性检验直至获得满意协变量平衡结果所需的工作量。然而，传统权重求解器缺乏可扩展性，难以在Snap等公司的大规模数据集上应用此类方法。为突破这些限制并提升计算效率，本文针对两种平衡方法——熵平衡法与MicroSynth——提出了可扩展算法DistEB和DistMS。这些求解器具有线性时间复杂度，且可便捷地在Spark、Hive等分布式计算框架中实现。我们研究了不同规模（处理组单位数达百万级、协变量达487个）下平衡方法的特性，发现随着样本量增大，因果效应估计的偏误和方差均显著降低。这一结果强调了在大规模数据集中应用平衡方法的重要性。我们将平衡方法与合成控制框架相结合，在Snap公司部署了一套端到端的因果影响估计系统。