In this work, we propose a novel Parameter-Efficient Fine-Tuning (PEFT) approach based on Gaussian Graphical Models (GGMs), marking the first application of GGMs to PEFT tasks, to the best of our knowledge. The proposed method utilizes the $\ell_{2,g}$-norm to effectively select critical parameters and capture global dependencies. The resulting non-convex optimization problem is efficiently solved using a Block Coordinate Descent (BCD) algorithm. Experimental results on the GLUE benchmark [24] for fine-tuning RoBERTa-Base [18] demonstrate the effectiveness of the proposed approach, achieving competitive performance with significantly fewer trainable parameters. The code for this work is available at: https://github.com/jzheng20/Course projects.git.

翻译：本文提出了一种基于高斯图模型（GGMs）的新型参数高效微调（PEFT）方法。据我们所知，这是GGMs在PEFT任务中的首次应用。所提方法利用$\ell_{2,g}$范数有效选择关键参数并捕获全局依赖关系。由此产生的非凸优化问题通过块坐标下降（BCD）算法高效求解。在GLUE基准[24]上对RoBERTa-Base[18]进行微调的实验结果表明，该方法能以显著更少的可训练参数达到具有竞争力的性能，验证了其有效性。本工作的代码公开于：https://github.com/jzheng20/Course projects.git。