Modeling true world data-generating processes lies at the heart of empirical science. Structural Causal Models (SCMs) and their associated Directed Acyclic Graphs (DAGs) provide an increasingly popular answer to such problems by defining the causal generative process that transforms random noise into observations. However, learning them from observational data poses an ill-posed and NP-hard inverse problem in general. In this work, we propose a new and equivalent formalism that do not require DAGs to describe them, viewed as fixed-point problems on the causally ordered variables, and show three important cases where they can be uniquely recovered given the topological ordering (TO). To the best of our knowledge, we obtain the most general recovery results when the TO is known. Based on our theoretical findings, we design a two-stage causal generative model that first infers the causal order from observations in a zero-shot manner, thus by-passing the search, and then learns the generative fixed-point SCM on the ordered variables. To infer TOs from observations, we propose to amortize the learning of TOs on generated datasets by sequentially predicting the leaves of graphs seen during training. To learn fixed-point SCMs, we design a transformer-based architecture that exploits a new attention mechanism enabling the modeling of causal structures, and show that this parameterization is consistent with our formalism. Finally, we conduct an extensive evaluation of each method individually, and show that when combined, our model outperforms various baselines on generated out-of-distribution problems.

翻译：对真实世界数据生成过程进行建模是实证科学的核心。结构因果模型（SCM）及其关联的有向无环图（DAG）通过定义将随机噪声转化为观测数据的因果生成过程，为这类问题提供了日益流行的解决方案。然而，从观测数据中学习这些模型通常是一个不适定且NP难的逆问题。本文提出了一种无需DAG描述的新等价形式，将其视为因果序变量上的定点问题，并展示了三种重要情形下，在给定拓扑序（TO）时能够唯一恢复该模型。据我们所知，当TO已知时，我们获得了最通用的恢复结果。基于理论发现，我们设计了一个两阶段因果生成模型：首先以零样本方式从观测数据中推断因果顺序（从而绕开搜索过程），然后在有序变量上学习生成式定点SCM。为从观测数据推断TO，我们提出通过顺序预测训练中见过的图的叶节点，在生成数据集上摊销TO的学习过程。为学习定点SCM，我们设计了一种基于Transformer的架构，利用新型注意力机制实现因果结构的建模，并证明该参数化与我们的形式化框架一致。最后，我们对每种方法进行单独评估，并表明当两者结合时，该模型在生成的分布外问题上优于多种基线方法。