Emerging Large Language Models (LLMs) like GPT-4 have revolutionized Natural Language Processing (NLP), showing potential in traditional tasks such as Named Entity Recognition (NER). Our study explores a three-phase training strategy that harnesses GPT-4's capabilities to enhance the BERT model's performance on NER. Initially, GPT-4 annotates a subset of the CONLL2003 and additional BBC dataset without fine-tuning. We then train BERT using a mix of original and LLM-annotated data, analyzing the efficacy of LLM annotations against traditional methods. The second phase involves comparative experiments with different training regimens, assessing the synergy between distilled and original data. We observe that sequential strategies, particularly a simple mix of training first with distilled data followed by original data, significantly boost performance. In the third phase, we investigate various data blending techniques, including sigmoid and power decay functions, to optimize the training process further. Our results indicate that a strategic mix of distilled and original data markedly elevates the NER capabilities of BERT. Our approach presents a scalable methodology that reduces manual annotation costs and increases efficiency, making it especially pertinent in resource-limited and closed-network environments. The study concludes that while the 'Simple Mix' strategy yields the best results, understanding its underlying mechanisms requires further research. Future work will also focus on refining prompt designs and enhancing annotation selection processes, aiming to extend our methodology to diverse NLP tasks.

翻译：诸如GPT-4等新兴大型语言模型（LLMs）已彻底革新自然语言处理（NLP）领域，在命名实体识别（NER）等传统任务中展现出巨大潜力。本研究探索一种三阶段训练策略，借助GPT-4的能力提升BERT模型在NER任务上的性能。第一阶段，在不进行微调的前提下，使用GPT-4对CONLL2003子集及额外BBC数据集进行标注。随后，我们利用原始数据与大语言模型标注数据的混合集训练BERT，通过与传统方法对比分析大语言模型标注的有效性。第二阶段开展基于不同训练方案的对比实验，评估蒸馏数据与原始数据间的协同效应。我们发现，序贯策略（尤其是先使用蒸馏数据训练再引入原始数据的简单混合方法）能显著提升性能。第三阶段，我们深入探究包括Sigmoid函数和幂衰减函数在内的多种数据混合技术，进一步优化训练流程。实验结果表明，蒸馏数据与原始数据的策略性混合可显著增强BERT的NER能力。本方法提出了一种可扩展的方案，通过降低人工标注成本并提升效率，使其在资源受限及封闭网络环境中尤为适用。研究最终表明，尽管"简单混合"策略能取得最佳效果，但其底层机制仍需进一步探究。未来工作将聚焦优化提示设计、改进标注筛选流程，旨在将本方法推广至更多元化的NLP任务。