Large Language Models (LLMs), such as GPT-4 and Llama 2, show remarkable proficiency in a wide range of natural language processing (NLP) tasks. Despite their effectiveness, the high costs associated with their use pose a challenge. We present LlamBERT, a hybrid approach that leverages LLMs to annotate a small subset of large, unlabeled databases and uses the results for fine-tuning transformer encoders like BERT and RoBERTa. This strategy is evaluated on two diverse datasets: the IMDb review dataset and the UMLS Meta-Thesaurus. Our results indicate that the LlamBERT approach slightly compromises on accuracy while offering much greater cost-effectiveness.

翻译：大型语言模型（如GPT-4和Llama 2）在各类自然语言处理任务中展现出卓越性能。尽管效果显著，但其高昂的使用成本构成了挑战。本文提出LlamBERT混合方法，该方法利用大型语言模型对大规模未标注数据库中的小规模子集进行标注，并将标注结果用于微调BERT、RoBERTa等Transformer编码器。本研究在IMDb评论数据集与UMLS元词库两个异构数据集上评估该策略。结果表明，LlamBERT方法在精度略有降低的同时，实现了显著更高的成本效益。