Both standalone language models (LMs) as well as LMs within downstream-task systems have been shown to generate statements which are factually untrue. This problem is especially severe for low-resource languages, where training data is scarce and of worse quality than for high-resource languages. In this opinion piece, we argue that LMs in their current state will never be fully trustworthy in critical settings and suggest a possible novel strategy to handle this issue: by building LMs such that can cite their sources - i.e., point a user to the parts of their training data that back up their outputs. We first discuss which current NLP tasks would or would not benefit from such models. We then highlight the expected benefits such models would bring, e.g., quick verifiability of statements. We end by outlining the individual tasks that would need to be solved on the way to developing LMs with the ability to cite. We hope to start a discussion about the field's current approach to building LMs, especially for low-resource languages, and the role of the training data in explaining model generations.

翻译：无论是独立的语言模型（LMs）还是下游任务系统中的LMs，都被证明会生成与事实不符的陈述。这一问题在低资源语言中尤为严重，因为这些语言的训练数据稀缺且质量低于高资源语言。在这篇评论文章中，我们认为当前状态的语言模型在关键场景中永远无法完全可信，并提出了一种可能的创新策略来解决这一问题：构建能够引用其来源的LMs——即向用户指出训练数据中支持其输出的部分。我们首先讨论了当前哪些NLP任务会从这类模型中受益或不受益。随后，我们强调了这类模型可能带来的预期好处，例如对陈述的快速可验证性。最后，我们概述了在开发具备引用能力的LMs过程中需要逐一解决的关键任务。我们期望借此引发学界对当前构建LMs方法的讨论，特别是针对低资源语言，以及训练数据在解释模型生成内容中的作用。