Large language models (LLMs) exhibit substantial performance disparities across languages, particularly between high- and low-resource settings. We propose a framework for improving performance in underrepresented languages while preserving general-purpose capabilities via targeted fine-tuning of sparse, language-associated subnetworks. Our approach identifies language-relevant neurons using Language Activation Probability Entropy (LAPE), an information-theoretic metric that reliably captures language-specific activation patterns, and fine-tunes only the corresponding weights. Experiments on Llama-3.1-8B, Mistral-Nemo-12B, and Aya-Expanse-8B across 12 mid- and low-resource languages show that our method consistently outperforms full fine-tuning, FFN-only fine-tuning, LoRA, IA^3, and random-subset baselines while updating only 0.2-1% of model parameters. We further show that sparse, neuron-targeted fine-tuning can inject new language capabilities without catastrophic forgetting, with potential applicability to other model capabilities. Mechanistic analyses of weight updates and internal representations reveal asymmetric roles of FFN projections in language adaptation and improved cross-lingual alignment. Finally, we release language neuron sets for over 100 languages together with our adaptation pipeline, enabling a cost-effective path for extending LLMs to underrepresented languages.

翻译：大型语言模型（LLMs）在不同语言间存在显著的性能差异，尤其在高低资源语言之间更为明显。本文提出一种通过针对性微调稀疏的语言关联子网络来提升低资源语言性能并保持通用能力的框架。该方法利用语言激活概率熵（LAPE）——一种能可靠捕捉语言特异性激活模式的信息论度量——识别语言相关神经元，并仅对相应权重进行微调。在Llama-3.1-8B、Mistral-Nemo-12B和Aya-Expanse-8B模型上对12种中低资源语言的实验表明，本方法在仅更新0.2-1%模型参数的情况下，持续优于全参数微调、仅FFN微调、LoRA、IA^3及随机子网络基线。我们进一步证明，稀疏的神经元定向微调能够注入新的语言能力而不会产生灾难性遗忘，该方法对其他模型能力也具有潜在适用性。对权重更新和内部表征的机制分析揭示了FFN投影在语言适应中的非对称作用以及跨语言对齐的改进。最后，我们发布了涵盖100多种语言的神经元集合及适配流程，为将LLMs扩展至低资源语言提供了经济有效的路径。