Large Language Models (LLMs), including GPT-x and LLaMA2, have achieved remarkable performance in multiple Natural Language Processing (NLP) tasks. Under the premise that protein sequences constitute the protein language, Protein Large Language Models (ProLLMs) trained on protein corpora excel at de novo protein sequence generation. However, as of now, unlike LLMs in NLP, no ProLLM is capable of multiple tasks in the Protein Language Processing (PLP) field. This prompts us to delineate the inherent limitations in current ProLLMs: (i) the lack of natural language capabilities, (ii) insufficient instruction understanding, and (iii) high training resource demands. To address these challenges, we introduce a training framework to transform any general LLM into a ProLLM capable of handling multiple PLP tasks. Specifically, our framework utilizes low-rank adaptation and employs a two-stage training approach, and it is distinguished by its universality, low overhead, and scalability. Through training under this framework, we propose the ProLLaMA model, the first known ProLLM to handle multiple PLP tasks simultaneously. Experiments show that ProLLaMA achieves state-of-the-art results in the unconditional protein sequence generation task. In the controllable protein sequence generation task, ProLLaMA can design novel proteins with desired functionalities. In the protein property prediction task, ProLLaMA achieves nearly 100\% accuracy across many categories. The latter two tasks are beyond the reach of other ProLLMs. Code is available at \url{https://github.com/Lyu6PosHao/ProLLaMA}.

翻译：大型语言模型（LLMs），包括GPT-x和LLaMA2，已在多项自然语言处理任务中取得显著成果。基于蛋白质序列构成蛋白质语言的假设，在蛋白质语料上训练的蛋白质大语言模型在从头生成蛋白质序列方面表现优异。然而，截至目前，与自然语言处理领域的LLMs不同，尚无蛋白质大语言模型能够在蛋白质语言处理领域执行多项任务。这促使我们揭示当前蛋白质大语言模型的固有局限性：（i）缺乏自然语言能力，（ii）指令理解不足，（iii）训练资源需求高。为应对这些挑战，我们提出一种训练框架，可将任意通用LLM转化为能处理多项蛋白质语言处理任务的蛋白质大语言模型。具体而言，该框架采用低秩适应技术并执行两阶段训练方法，具有通用性、低开销和可扩展性。在此框架下训练的ProLLaMA模型，是首个已知能同时处理多项蛋白质语言处理任务的蛋白质大语言模型。实验表明，ProLLaMA在无条件蛋白质序列生成任务中达到最优结果；在可控蛋白质序列生成任务中，可设计具有所需功能的新型蛋白质；在蛋白质属性预测任务中，对多个类别的预测准确率接近100%。后两项任务超出了其他蛋白质大语言模型的能力范围。代码已公开于\url{https://github.com/Lyu6PosHao/ProLLaMA}。