One of the key problems in multi-label text classification is how to take advantage of the correlation among labels. However, it is very challenging to directly model the correlations among labels in a complex and unknown label space. In this paper, we propose a Label Mask multi-label text classification model (LM-MTC), which is inspired by the idea of cloze questions of language model. LM-MTC is able to capture implicit relationships among labels through the powerful ability of pre-train language models. On the basis, we assign a different token to each potential label, and randomly mask the token with a certain probability to build a label based Masked Language Model (MLM). We train the MTC and MLM together, further improving the generalization ability of the model. A large number of experiments on multiple datasets demonstrate the effectiveness of our method.

翻译：多标签文本分类的关键问题之一是如何利用标签之间的相关性。然而，在复杂且未知的标签空间中直接对标签相关性进行建模极具挑战性。本文受语言模型完形填空任务思想的启发，提出了一种标签掩码多标签文本分类模型（LM-MTC）。该模型能够通过预训练语言模型的强大能力捕捉标签间的隐式关联关系。在此基础上，我们为每个潜在标签分配不同的词元，并以一定概率随机掩码这些词元，从而构建基于标签的掩码语言模型（MLM）。通过联合训练多标签文本分类任务与掩码语言模型，进一步提升了模型的泛化能力。在多个数据集上的大量实验验证了我们方法的有效性。