Despite rapid advancement in the field of Constrained Natural Language Generation, little time has been spent on exploring the potential of language models which have had their vocabularies lexically, semantically, and/or phonetically constrained. We find that most language models generate compelling text even under significant constraints. We present a simple and universally applicable technique for modifying the output of a language model by compositionally applying filter functions to the language models vocabulary before a unit of text is generated. This approach is plug-and-play and requires no modification to the model. To showcase the value of this technique, we present an easy to use AI writing assistant called Constrained Text Generation Studio (CTGS). CTGS allows users to generate or choose from text with any combination of a wide variety of constraints, such as banning a particular letter, forcing the generated words to have a certain number of syllables, and/or forcing the words to be partial anagrams of another word. We introduce a novel dataset of prose that omits the letter e. We show that our method results in strictly superior performance compared to fine-tuning alone on this dataset. We also present a Huggingface space web-app presenting this technique called Gadsby. The code is available to the public here: https://github.com/Hellisotherpeople/Constrained-Text-Generation-Studio

翻译：尽管约束自然语言生成领域发展迅速，但针对词汇在词汇、语义和/或语音层面受到约束的语言模型的潜力探索尚不充分。我们发现，即使在显著约束条件下，大多数语言模型仍能生成引人入胜的文本。我们提出一种简单且普遍适用的技术，通过在文本生成前对语言模型的词汇表以组合方式应用过滤函数来修改其输出。该方法即插即用，无需修改模型本身。为展示该技术的价值，我们开发了一款易于使用的AI写作辅助工具——约束文本生成工作室（CTGS）。CTGS允许用户生成或选择符合多种约束组合的文本，例如禁止使用特定字母、强制生成单词具有特定音节数，以及/或强制单词为另一单词的部分变位词。我们引入了一个新颖的散文数据集，其中省略了字母e。实验证明，在此数据集上，我们的方法相比单独微调具有严格更优的性能。我们还推出了一款名为Gadsby的Huggingface空间网页应用来展示该技术。代码已公开于：https://github.com/Hellisotherpeople/Constrained-Text-Generation-Studio