Unsupervised constrained text generation aims to generate text under a given set of constraints without any supervised data. Current state-of-the-art methods stochastically sample edit positions and actions, which may cause unnecessary search steps. In this paper, we propose PMCTG to improve effectiveness by searching for the best edit position and action in each step. Specifically, PMCTG extends perturbed masking technique to effectively search for the most incongruent token to edit. Then it introduces four multi-aspect scoring functions to select edit action to further reduce search difficulty. Since PMCTG does not require supervised data, it could be applied to different generation tasks. We show that under the unsupervised setting, PMCTG achieves new state-of-the-art results in two representative tasks, namely keywords-to-sentence generation and paraphrasing.
翻译:无监督约束文本生成旨在无需任何监督数据的情况下,在给定约束条件下生成文本。当前最先进的方法随机采样编辑位置和动作,可能导致不必要的搜索步骤。本文提出PMCTG方法,通过在每个步骤中搜索最佳编辑位置和动作来提高效率。具体而言,PMCTG扩展了扰动掩码技术,以有效搜索最不协调的待编辑词元。随后引入四种多维度评分函数来选择编辑动作,进一步降低搜索难度。由于PMCTG无需监督数据,因此可应用于不同的生成任务。结果表明,在无监督设置下,PMCTG在关键词到句子生成和释义生成两项代表性任务中均取得了新的最佳结果。