We investigate strategies for adapting small, efficient language models to Faroese, a low-resource North Germanic language. Starting from English-pretrained models, we apply continued pre-training on related Scandinavian languages -- individually or combined via model merging -- before fine-tuning on Faroese. We compare full fine-tuning with parameter-efficient adaptation via LoRA, assessing their effects on general language modeling performance, linguistic accuracy, and text comprehension. To address the lack of existing Faroese evaluation resources, we construct two new minimal-pair probing benchmarks, one for linguistic acceptability and one for text comprehension, and complement them with human evaluations conducted by native Faroese linguists. Our results show that transfer from related languages is essential, but the optimal source language is task-dependent: Icelandic improves linguistic accuracy, while Danish boosts reading comprehension. The choice of adaptation method likewise depends on the target task: LoRA yields stronger linguistic acceptability and marginally higher human evaluation scores, whereas full fine-tuning produces better comprehension performance and more robust downstream fine-tuning. Merging multiple related languages under full fine-tuning (but not LoRA) improves general language modeling, though its benefits in the linguistic acceptability and comprehension probes are less consistent.

翻译：本文探究了将小型高效语言模型适配至低资源北日耳曼语种——法罗语的策略。以英语预训练模型为起点，我们通过在相关斯堪的纳维亚语言上进行持续预训练（采用模型合并技术分别或组合处理），再针对法罗语进行微调。我们比较了全参数微调与基于LoRA的参数高效适配方法，评估其对通用语言建模性能、语言准确性和文本理解的影响。鉴于现有法罗语评估资源匮乏，我们构建了两个全新的最小对探针基准（语法可接受性与文本理解测试），并辅之以母语为法罗语的语言学家开展的人工评估。实验结果表明：从关联语言迁移至关重要，但最优源语言取决于具体任务——冰岛语提升语言准确性，而丹麦语增强阅读理解。适配方法的选择同样与目标任务相关：LoRA在语法可接受性及人工评分中表现更优，全参数微调则在理解性能与下游任务微调稳健性上更胜一筹。在全参数微调（而非LoRA）场景下合并多种关联语言可改善通用语言建模效果，但该方法在语法可接受性与理解探针任务中的效益尚存波动。