Recent language models have been improved by the addition of external memory. Nearest neighbor language models retrieve similar contexts to assist in word prediction. The addition of locality levels allows a model to learn how to weight neighbors based on their relative location to the current text in source documents, and have been shown to further improve model performance. Nearest neighbor models have been explored for controllable generation but have not examined the use of locality levels. We present a novel approach for this purpose and evaluate it using automatic and human evaluation on politeness, formality, supportiveness, and toxicity textual data. We find that our model is successfully able to control style and provides a better fluency-style trade-off than previous work.

翻译：近期语言模型通过引入外部记忆得到了改进。最近邻语言模型通过检索相似上下文来辅助词预测。局部性层级的加入使模型能够根据源文档中邻居相对于当前文本的位置来学习如何对邻居进行加权，并已被证明能进一步提升模型性能。最近邻模型在可控生成领域已有探索，但尚未研究局部性层级的使用。我们针对这一目的提出了一种新颖方法，并在礼貌性、正式性、支持性和毒性文本数据上，通过自动评估和人工评估进行验证。结果表明，我们的模型能够成功控制风格，并在流畅性与风格平衡方面优于先前的工作。