Graph generation has been dominated by autoregressive models due to their simplicity and effectiveness, despite their sensitivity to ordering. Yet diffusion models have garnered increasing attention, as they offer comparable performance while being permutation-invariant. Current graph diffusion models generate graphs in a one-shot fashion, but they require extra features and thousands of denoising steps to achieve optimal performance. We introduce PARD, a Permutation-invariant Auto Regressive Diffusion model that integrates diffusion models with autoregressive methods. PARD harnesses the effectiveness and efficiency of the autoregressive model while maintaining permutation invariance without ordering sensitivity. Specifically, we show that contrary to sets, elements in a graph are not entirely unordered and there is a unique partial order for nodes and edges. With this partial order, PARD generates a graph in a block-by-block, autoregressive fashion, where each block's probability is conditionally modeled by a shared diffusion model with an equivariant network. To ensure efficiency while being expressive, we further propose a higher-order graph transformer, which integrates transformer with PPGN. Like GPT, we extend the higher-order graph transformer to support parallel training of all blocks. Without any extra features, PARD achieves state-of-the-art performance on molecular and non-molecular datasets, and scales to large datasets like MOSES containing 1.9M molecules. Pard is open-sourced at https://github.com/LingxiaoShawn/Pard.

翻译：图生成领域长期以来由自回归模型主导，这得益于其简单性和有效性，尽管它们对排序敏感。然而，扩散模型正获得越来越多的关注，因为它们提供了可比的性能且具有置换不变性。当前的图扩散模型以一次性方式生成图，但需要额外的特征和数千个去噪步骤才能达到最佳性能。我们提出了PARD，一种置换不变自回归扩散模型，它将扩散模型与自回归方法相结合。PARD利用了自回归模型的有效性和效率，同时保持了置换不变性而无需排序敏感性。具体而言，我们证明与集合不同，图中的元素并非完全无序，节点和边存在唯一的偏序关系。利用这种偏序关系，PARD以逐块自回归的方式生成图，其中每个块的概率由一个共享的扩散模型通过等变网络进行条件建模。为了在保持表达力的同时确保效率，我们进一步提出了一种高阶图Transformer，它将Transformer与PPGN相结合。类似于GPT，我们扩展了高阶图Transformer以支持所有块的并行训练。在无需任何额外特征的情况下，PARD在分子和非分子数据集上实现了最先进的性能，并可扩展到包含190万个分子的大型数据集如MOSES。Pard已在https://github.com/LingxiaoShawn/Pard开源。