Text classification of unseen classes is a challenging Natural Language Processing task and is mainly attempted using two different types of approaches. Similarity-based approaches attempt to classify instances based on similarities between text document representations and class description representations. Zero-shot text classification approaches aim to generalize knowledge gained from a training task by assigning appropriate labels of unknown classes to text documents. Although existing studies have already investigated individual approaches to these categories, the experiments in literature do not provide a consistent comparison. This paper addresses this gap by conducting a systematic evaluation of different similarity-based and zero-shot approaches for text classification of unseen classes. Different state-of-the-art approaches are benchmarked on four text classification datasets, including a new dataset from the medical domain. Additionally, novel SimCSE and SBERT-based baselines are proposed, as other baselines used in existing work yield weak classification results and are easily outperformed. Finally, the novel similarity-based Lbl2TransformerVec approach is presented, which outperforms previous state-of-the-art approaches in unsupervised text classification. Our experiments show that similarity-based approaches significantly outperform zero-shot approaches in most cases. Additionally, using SimCSE or SBERT embeddings instead of simpler text representations increases similarity-based classification results even further.

翻译：未见类别文本分类是一项具有挑战性的自然语言处理任务，主要采用两种不同方法。基于相似度的方法尝试根据文本文档表示与类别描述表示之间的相似性对实例进行分类。零样本文本分类方法旨在通过为文本文档分配未知类别的适当标签，从训练任务中泛化知识。尽管现有研究已分别探讨了这两类方法，但文献中的实验缺乏一致性比较。本文通过系统评估基于相似度和零样本方法在未见类别文本分类中的表现，填补了这一空白。研究在四个文本分类数据集（包括一个来自医学领域的新数据集）上对多种最先进方法进行了基准测试。此外，提出了新型SimCSE和基于SBERT的基线模型，因为现有工作中使用的其他基线模型分类结果较弱且易被超越。最后，提出了基于相似度的新型Lbl2TransformerVec方法，其在无监督文本分类中优于先前最先进方法。实验表明，基于相似度的方法在大多数情况下显著优于零样本方法。此外，使用SimCSE或SBERT嵌入替代简单文本表示，能进一步提升基于相似度分类的结果。