The fine-tuning of Large Language Models (LLMs) has enabled them to recently achieve milestones in natural language processing applications. The emergence of ever larger LLMs has paved the way for more efficient fine-tuning methods. Among these, the Low-Rank Adaptation (LoRA) method keeps most of the weights of the pre-trained LLM frozen while introducing a low-rank decomposition of the weight matrix, enabling the tuning of only a very small proportion of the network. The performance on downstream tasks of models fine-tuned with LoRA heavily relies on a set of hyperparameters including the rank of the decomposition. In this work, we investigate the choice of these hyperparameters through two main blackbox optimization (BBO) techniques. We examine the whole pipeline of performing fine-tuning and validation on a pre-trained LLM as a blackbox and efficiently explore the space of hyperparameters with the \nomad algorithm, achieving a boost in performance and human alignment of the tuned model.

翻译：大规模语言模型（LLM）的微调使其近期在自然语言处理应用中取得了里程碑式进展。日益庞大的LLM催生了更高效的微调方法。其中，低秩自适应（LoRA）方法在保持预训练LLM大部分权重冻结的同时，引入权重矩阵的低秩分解，仅需调整网络中极小比例的参数。使用LoRA微调的模型在下游任务上的性能严重依赖于包括分解秩在内的一组超参数。本研究通过两种主流的黑盒优化（BBO）技术探究这些超参数的选择。我们将预训练LLM上执行微调与验证的完整流程视为一个黑盒，并利用\nomad算法高效探索超参数空间，从而提升调优模型的性能与人类对齐度。