There is abundant observational data in the software engineering domain, whereas running large-scale controlled experiments is often practically impossible. Thus, most empirical studies can only report statistical correlations -- instead of potentially more insightful and robust causal relations. To support analyzing purely observational data for causal relations, and to assess any differences between purely predictive and causal models of the same data, this paper discusses some novel techniques based on structural causal models (such as directed acyclic graphs of causal Bayesian networks). Using these techniques, one can rigorously express, and partially validate, causal hypotheses; and then use the causal information to guide the construction of a statistical model that captures genuine causal relations -- such that correlation does imply causation. We apply these ideas to analyzing public data about programmer performance in Code Jam, a large world-wide coding contest organized by Google every year. Specifically, we look at the impact of different programming languages on a participant's performance in the contest. While the overall effect associated with programming languages is weak compared to other variables -- regardless of whether we consider correlational or causal links -- we found considerable differences between a purely associational and a causal analysis of the very same data. The takeaway message is that even an imperfect causal analysis of observational data can help answer the salient research questions more precisely and more robustly than with just purely predictive techniques -- where genuine causal effects may be confounded.

翻译：软件工程领域存在大量观测数据，而大规模受控实验往往在实践上难以实现。因此，多数实证研究只能报告统计相关性——而非可能更具洞察力和稳健性的因果关系。为了支持对纯观测数据进行因果关系的分析，并评估同一数据集上纯预测模型与因果模型之间的差异，本文讨论了基于结构因果模型（如因果贝叶斯网络的有向无环图）的若干新技术。通过这些技术，可以严格表述并部分验证因果假设，进而利用因果信息指导构建统计模型，以捕获真正的因果关系——从而使相关性蕴含因果性。我们将这些思想应用于分析谷歌每年举办的大型全球编程竞赛Code Jam中程序员表现的公开数据。具体而言，我们研究了不同编程语言对参赛者在竞赛中表现的影响。尽管无论考虑关联性还是因果性链接，编程语言相关的总体效应相对于其他变量均较弱，但我们发现同一数据集的纯关联分析与因果分析之间存在显著差异。主要启示是：即使对观测数据进行不完美的因果分析，也能比纯预测技术更精确、更稳健地回答关键研究问题——而纯预测技术中真实的因果效应可能被混淆。