Biomedical literature is a rapidly expanding field of science and technology. Classification of biomedical texts is an essential part of biomedicine research, especially in the field of biology. This work proposes the fine-tuned DistilBERT, a methodology-specific, pre-trained generative classification language model for mining biomedicine texts. The model has proven its effectiveness in linguistic understanding capabilities and has reduced the size of BERT models by 40\% but by 60\% faster. The main objective of this project is to improve the model and assess the performance of the model compared to the non-fine-tuned model. We used DistilBert as a support model and pre-trained on a corpus of 32,000 abstracts and complete text articles; our results were impressive and surpassed those of traditional literature classification methods by using RNN or LSTM. Our aim is to integrate this highly specialised and specific model into different research industries.

翻译：生物医学文献是科学技术中一个快速发展的领域。生物医学文本分类是生物医学研究的重要组成部分，尤其在生物学领域。本文提出了经过微调的DistilBERT——一种针对特定方法学、预训练的分类生成语言模型，用于挖掘生物医学文本。该模型在语言理解能力方面证明了其有效性，并将BERT模型的规模缩小了40%，同时速度提升了60%。本项目的主要目标是改进模型，并评估其与未经微调的模型相比的性能表现。我们使用DistilBERT作为支撑模型，并在包含32,000篇摘要和全文文章的语料库上进行预训练；我们的结果令人印象深刻，超越了使用RNN或LSTM的传统文献分类方法。我们的目标是将这一高度专业化的模型整合到不同的研究行业中。