Machine learning methods estimate flexible relationships between confounders, treatments and outcomes, making unconfoundedness assumptions in causal analysis more palatable. How successful are these approaches in recovering ground truth baselines? We analyze new data covering an experimental rollout of a software product feature for one group of users and a voluntary opt-in to the same feature for another similar group. Recovering causal effects from nonrandomized data is feasible--but only with careful modeling choices. We offer best practices for more credible treatment effect estimation in modern, high-dimensional datasets and evaluate which tests provide meaningful signals about the reliability of estimated effects.

翻译：暂无翻译