This paper presents a systematic overview and comparison of parameter-efficient fine-tuning methods covering over 40 papers published between February 2019 and February 2023. These methods aim to resolve the infeasibility and impracticality of fine-tuning large language models by only training a small set of parameters. We provide a taxonomy that covers a broad range of methods and present a detailed method comparison with a specific focus on real-life efficiency and fine-tuning multibillion-scale language models.

翻译：本文系统概述并比较了参数高效微调方法，涵盖2019年2月至2023年2月间发表的40余篇论文。这些方法旨在通过仅训练少量参数来解决大型语言模型微调不可行且不切实际的问题。我们提出了一种涵盖多种方法的分类体系，并进行了详细的方法比较，特别关注实际效率及对数十亿规模语言模型的微调。