This paper delves into the pressing need in Parameter-Efficient Fine-Tuning (PEFT) for Large Language Models (LLMs). While LLMs possess remarkable capabilities, their extensive parameter requirements and associated computational demands hinder their practicality and scalability for real-world applications. Our position paper highlights current states and the necessity of further studying into the topic, and recognizes significant challenges and open issues that must be addressed to fully harness the powerful abilities of LLMs. These challenges encompass novel efficient PEFT architectures, PEFT for different learning settings, PEFT combined with model compression techniques, and the exploration of PEFT for multi-modal LLMs. By presenting this position paper, we aim to stimulate further research and foster discussions surrounding more efficient and accessible PEFT for LLMs.

翻译：本文深入探讨了大语言模型（LLMs）中参数高效微调（PEFT）的迫切需求。尽管大语言模型具备卓越能力，但其庞大的参数需求及相应的计算开销限制了其在现实应用中的实用性与可扩展性。本立场论文阐述了该领域的研究现状及进一步研究的必要性，并指出了充分释放大语言模型强大能力所必须应对的关键挑战与开放性问题。这些挑战涵盖新型高效PEFT架构、不同学习场景下的PEFT、PEFT与模型压缩技术的结合，以及面向多模态大语言模型的PEFT探索。通过呈现本立场论文，我们旨在激发对更高效、更易用的大语言模型PEFT方法的进一步研究，并促进相关讨论。