Understanding the causal relationships among the variables of a system is paramount to explain and control its behaviour. Inferring the causal graph from observational data without interventions, however, requires a lot of strong assumptions that are not always realistic. Even for domain experts it can be challenging to express the causal graph. Therefore, metrics that quantitatively assess the goodness of a causal graph provide helpful checks before using it in downstream tasks. Existing metrics provide an absolute number of inconsistencies between the graph and the observed data, and without a baseline, practitioners are left to answer the hard question of how many such inconsistencies are acceptable or expected. Here, we propose a novel consistency metric by constructing a surrogate baseline through node permutations. By comparing the number of inconsistencies with those on the surrogate baseline, we derive an interpretable metric that captures whether the DAG fits significantly better than random. Evaluating on both simulated and real data sets from various domains, including biology and cloud monitoring, we demonstrate that the true DAG is not falsified by our metric, whereas the wrong graphs given by a hypothetical user are likely to be falsified.

翻译：理解系统中变量之间的因果关系对于解释和控制其行为至关重要。然而，在没有干预的情况下从观测数据推断因果图需要大量强有力的假设，这些假设并不总是现实的。即使是领域专家，表达因果图也可能具有挑战性。因此，在将因果图用于下游任务之前，能够定量评估其质量的指标提供了有用的检验。现有指标提供了图与观测数据之间不一致的绝对数量，但缺乏基线，实践者难以回答多少此类不一致是可接受或预期的。在此，我们通过节点排列构建代理基线，提出了一种新的一致性指标。通过将不一致数量与代理基线进行比较，我们推导出一个可解释的指标，用于捕捉有向无环图（DAG）是否显著优于随机拟合。通过在包括生物学和云监控在内的多个领域的模拟和真实数据集上进行评估，我们证明了该指标不会证伪真实的DAG，而假设用户提供的错误图则很可能被证伪。