This review surveys the rapid evolution of Meta AI's LLaMA (Large Language Model Meta AI) series - from LLaMA 1 through LLaMA 4 and the specialized parameter-efficient fine-tuning (PEFT) methods developed for these models. We first describe the LLaMA family of foundation models (7B-65B to 288B parameters), their architectures (including native multimodal and Mixtureof-Experts variants), and key performance characteristics. We then describe and discuss the concept of PEFT, which adapts large pre-trained models by updating only a small subset of parameters, and review five PEFT methods that have been applied to LLaMA: LoRA (Low-Rank Adaptation), LLaMA-Adapter V1 and V2, LLaMA-Excitor, and QLoRA (Quantized LoRA). We discuss each method's mechanism, parameter savings, and example application to LLaMA (e.g., instruction tuning, multimodal tasks). We provide structured discussion and analysis of model and adapter architectures, parameter counts, and benchmark results (including examples where fine-tuned LLaMA models outperform larger baselines). Finally, we examine real-world use cases where LLaMA-based models and PEFT have been successfully applied (e.g., legal and medical domains), and we discuss ongoing challenges and future research directions (such as scaling to even larger contexts and improving robustness). This survey paper provides a one-stop resource for ML researchers and practitioners interested in LLaMA models and efficient fine-tuning strategies.

翻译：本文综述了Meta AI的LLaMA（大型语言模型Meta AI）系列从LLaMA 1到LLaMA 4的快速演进历程，以及为这些模型开发的专用参数高效微调方法。我们首先描述了LLaMA系列基础模型（参数量7B-65B至288B）、其架构（包括原生多模态与专家混合变体）及关键性能特征。随后阐述并讨论了参数高效微调的概念——该方法仅更新预训练大模型的少量参数子集以实现适配，并系统回顾了已应用于LLaMA的五种PEFT方法：LoRA（低秩适配）、LLaMA-Adapter V1/V2、LLaMA-Excitor以及QLoRA（量化LoRA）。我们探讨了每种方法的机理、参数节约效益及在LLaMA上的典型应用案例（如指令微调、多模态任务）。通过对模型与适配器架构、参数量级及基准测试结果（包括微调后LLaMA模型超越更大规模基线的实例）进行结构化讨论与分析，系统梳理了该领域进展。最后，我们考察了基于LLaMA的模型与PEFT技术在实际场景中的成功应用案例（如法律与医疗领域），并探讨了当前面临的挑战与未来研究方向（例如扩展至更大上下文窗口、提升模型鲁棒性等）。本综述为关注LLaMA模型与高效微调策略的机器学习研究者与实践者提供了系统性的参考资料。