With the advent of strong pre-trained natural language processing models like BERT, DeBERTa, MiniLM, T5, the data requirement for industries to fine-tune these models to their niche use cases has drastically reduced (typically to a few hundred annotated samples for achieving a reasonable performance). However, the availability of even a few hundred annotated samples may not always be guaranteed in low resource domains like automotive, which often limits the usage of such deep learning models in an industrial setting. In this paper we aim to address the challenge of fine-tuning such pre-trained models with only a few annotated samples, also known as Few-shot learning. Our experiments focus on evaluating the performance of a diverse set of algorithms and methodologies to achieve the task of classifying BOSCH automotive domain textual software requirements into 3 categories, while utilizing only 15 annotated samples per category for fine-tuning. We find that while SciBERT and DeBERTa based models tend to be the most accurate at 15 training samples, their performance improvement scales minimally as the number of annotated samples is increased to 50 in comparison to Siamese and T5 based models.

翻译：随着BERT、DeBERTa、MiniLM、T5等强预训练自然语言处理模型的出现，工业界将这些模型微调到特定用例的数据需求已大幅降低（通常仅需数百个标注样本即可获得合理性能）。然而，在汽车等低资源领域，即使数百个标注样本也未必总能获得，这通常限制了此类深度学习模型在工业场景中的应用。本文旨在解决仅用少量标注样本微调这类预训练模型（即小样本学习）的挑战。我们的实验聚焦于评估多种算法与方法的性能，以完成博世汽车领域文本软件需求的三分类任务，且每个类别仅使用15个标注样本进行微调。实验发现，尽管基于SciBERT和DeBERTa的模型在15个训练样本下表现最为准确，但当标注样本数增至50时，其性能提升幅度与基于孪生网络和T5的模型相比微乎其微。