This paper explores the integration of graph knowledge from linguistic ontologies into multilingual Large Language Models (LLMs) using adapters to improve performance for low-resource languages (LRLs) in sentiment analysis (SA) and named entity recognition (NER). Building upon successful parameter-efficient fine-tuning techniques, such as K-ADAPTER and MAD-X, we propose a similar approach for incorporating knowledge from multilingual graphs, connecting concepts in various languages with each other through linguistic relationships, into multilingual LLMs for LRLs. Specifically, we focus on eight LRLs -- Maltese, Bulgarian, Indonesian, Nepali, Javanese, Uyghur, Tibetan, and Sinhala -- and employ language-specific adapters fine-tuned on data extracted from the language-specific section of ConceptNet, aiming to enable knowledge transfer across the languages covered by the knowledge graph. We compare various fine-tuning objectives, including standard Masked Language Modeling (MLM), MLM with full-word masking, and MLM with targeted masking, to analyse their effectiveness in learning and integrating the extracted graph data. Through empirical evaluation on language-specific tasks, we assess how structured graph knowledge affects the performance of multilingual LLMs for LRLs in SA and NER, providing insights into the potential benefits of adapting language models for low-resource scenarios.

翻译：本文探讨了如何通过适配器将语言本体中的图知识整合到多语言大语言模型中，以提升低资源语言在情感分析和命名实体识别任务上的性能。在K-ADAPTER和MAD-X等成功的参数高效微调技术基础上，我们提出了一种类似的方法，用于将多语言知识图谱（通过语言关系连接不同语言的概念）中的知识融入面向低资源语言的多语言大语言模型。具体而言，我们聚焦于八种低资源语言——马耳他语、保加利亚语、印尼语、尼泊尔语、爪哇语、维吾尔语、藏语和僧伽罗语，并采用在从ConceptNet相应语言分区提取的数据上微调的语言特定适配器，旨在实现知识图谱所覆盖语言间的知识迁移。我们比较了多种微调目标，包括标准的掩码语言建模、全词掩码的掩码语言建模以及定向掩码的掩码语言建模，以分析它们在学习和整合所提取图数据方面的有效性。通过对语言特定任务的实证评估，我们评估了结构化图知识如何影响多语言大语言模型在低资源语言情感分析和命名实体识别上的性能，从而为低资源场景下语言模型适应的潜在优势提供了见解。