Pre-trained language models and other generative models have revolutionized NLP and beyond. However, these models tend to reproduce undesirable biases present in their training data. Also, they may overlook patterns that are important but challenging to capture. To address these limitations, researchers have introduced distributional control techniques. These techniques, not limited to language, allow controlling the prevalence (i.e., expectations) of any features of interest in the model's outputs. Despite their potential, the widespread adoption of these techniques has been hindered by the difficulty in adapting complex, disconnected code. Here, we present disco, an open-source Python library that brings these techniques to the broader public.
翻译:预训练语言模型及其他生成模型已彻底改变了自然语言处理及其他领域。然而,这些模型往往会复现其训练数据中存在的不良偏见。此外,它们可能忽略那些重要但难以捕捉的模式。为解决这些局限性,研究者引入了分布控制技术。这些技术不仅限于语言领域,还能控制模型输出中任何感兴趣特征的普遍性(即期望值)。尽管具有潜力,但这些技术的广泛采用一直受限于复杂且不连贯代码的适配难度。在此,我们提出disco,一个开源的Python库,旨在将这些技术推广至更广泛的用户群体。