Large Language Models (LLMs), often show strong performance on English tasks, while exhibiting limitations on other languages. What is an LLM's multilingual capability when it is trained only on certain languages? The underlying mechanism remains unclear. This study endeavors to examine the multilingual capability of LLMs from the vocabulary sharing perspective by conducting an exhaustive analysis across 101 languages. Through the investigation of the performance gap before and after embedding fine-tuning, we discovered four distinct quadrants. By delving into each quadrant we provide actionable and efficient guidelines for tuning these languages. Extensive experiments reveal that existing LLMs possess multilingual capabilities that surpass our expectations, and we can significantly improve the multilingual performance of LLMs based on these attributes of each quadrant~\footnote{\url{https://github.com/CONE-MT/Vocabulary-Sharing-Facilitates-Multilingualism}.}.

翻译：大型语言模型（LLMs）通常在英语任务上表现出色，但在其他语言上则存在局限性。当LLM仅在某些语言上训练时，其多语言能力究竟如何？其内在机制尚不明确。本研究旨在从词汇共享的角度，通过对101种语言进行详尽分析，探讨LLMs的多语言能力。通过考察嵌入微调前后的性能差距，我们发现了四个不同的象限。通过深入分析每个象限，我们为调整这些语言提供了可行且高效的指导原则。大量实验表明，现有LLMs的多语言能力超出了我们的预期，并且我们可以基于每个象限的这些特性显著提升LLMs的多语言性能~\footnote{\url{https://github.com/CONE-MT/Vocabulary-Sharing-Facilitates-Multilingualism}.}。