Generative models reliant on sequential autoregression have been at the forefront of language generation for an extensive period, particularly following the introduction of widely acclaimed transformers. Despite its excellent performance, there are always some issues that we face today. For example, problems such as hallucinations and getting trapped in a logic loop may occur. To enhance the performance of existing systems, this paper introduces a new method for generating sequences in natural language, which involves generating the targeted sentence in a tree-traversing order. The paper includes an illustration of the theoretical basis and validity of the approach, as well as a comparison of its fundamentals with the diffusion model in graphic generation. Finally, a module called SenTree is introduced for generating an approximating binary tree. It is already available at https://github.com/arklyg/sentree. Additionally, a joint training framework based on this approach is proposed, incorporating the intrinsics of generative adversarial networks.

翻译：依赖序列自回归的生成模型长期以来一直处于语言生成的前沿，尤其是在广受赞誉的Transformer架构被引入之后。尽管其性能卓越，但我们今天仍面临一些问题。例如，可能会出现幻觉和陷入逻辑循环等问题。为了提升现有系统的性能，本文提出了一种新的自然语言序列生成方法，该方法以树遍历顺序生成目标句子。本文阐述了该方法的理论基础和有效性，并将其基本原理与图像生成中的扩散模型进行了比较。最后，介绍了一个名为SenTree的模块，用于生成近似二叉树。该模块已在https://github.com/arklyg/sentree上发布。此外，本文还提出了一种基于此方法的联合训练框架，该框架融合了生成对抗网络的内在特性。