The information bottleneck (IB) principle has been proven effective in various NLP applications. The existing work, however, only used either generative or information compression models to improve the performance of the target task. In this paper, we propose to combine the two types of IB models into one system to enhance Named Entity Recognition (NER). For one type of IB model, we incorporate two unsupervised generative components, span reconstruction and synonym generation, into a span-based NER system. The span reconstruction ensures that the contextualised span representation keeps the span information, while the synonym generation makes synonyms have similar representations even in different contexts. For the other type of IB model, we add a supervised IB layer that performs information compression into the system to preserve useful features for NER in the resulting span representations. Experiments on five different corpora indicate that jointly training both generative and information compression models can enhance the performance of the baseline span-based NER system. Our source code is publicly available at https://github.com/nguyennth/joint-ib-models.

翻译：信息瓶颈（IB）原理已在多种自然语言处理应用中证明其有效性。然而，现有工作仅单独使用生成模型或信息压缩模型来提升目标任务性能。本文提出将两类IB模型融合到同一系统中以增强命名实体识别（NER）。针对第一类IB模型，我们在基于跨度的NER系统中引入两个无监督生成组件：跨度重构与同义词生成。跨度重构确保上下文跨度表示保留跨度信息，而同义词生成使同义词即使在不同上下文中也具有相似表示。针对第二类IB模型，我们在系统中添加有监督IB层执行信息压缩，以在生成的跨度表示中保留NER所需的特征。在五个不同语料库上的实验表明，联合训练生成模型与信息压缩模型可提升基线跨度NER系统的性能。我们的源代码已公开于https://github.com/nguyennth/joint-ib-models。