Large language models (LLMs) have demonstrated impressive capabilities across diverse languages. This study explores how LLMs handle multilingualism. Based on observed language ratio shifts among layers and the relationships between network structures and certain capabilities, we hypothesize the LLM's multilingual workflow ($\texttt{MWork}$): LLMs initially understand the query, converting multilingual inputs into English for task-solving. In the intermediate layers, they employ English for thinking and incorporate multilingual knowledge with self-attention and feed-forward structures, respectively. In the final layers, LLMs generate responses aligned with the original language of the query. To verify $\texttt{MWork}$, we introduce Parallel Language-specific Neuron Detection ($\texttt{PLND}$) to identify activated neurons for inputs in different languages without any labeled data. Using $\texttt{PLND}$, we validate $\texttt{MWork}$ through extensive experiments involving the deactivation of language-specific neurons across various layers and structures. Moreover, $\texttt{MWork}$ allows fine-tuning of language-specific neurons with a small dataset, enhancing multilingual abilities in a specific language without compromising others. This approach results in an average improvement of $3.6\%$ for high-resource languages and $2.3\%$ for low-resource languages across all tasks with just $400$ documents.

翻译：大型语言模型（LLMs）在多种语言中展现出了令人印象深刻的能力。本研究探讨了LLMs如何处理多语言问题。基于观察到的各层间语言比例变化以及网络结构与特定能力之间的关系，我们提出了LLM的多语言工作流程假设（$\texttt{MWork}$）：LLMs首先理解查询，将多语言输入转换为英语以进行任务求解。在中间层，它们使用英语进行思考，并分别通过自注意力结构和前馈结构融入多语言知识。在最后几层，LLMs生成与查询原始语言一致的回答。为了验证$\texttt{MWork}$，我们引入了并行语言特定神经元检测方法（$\texttt{PLND}$），无需任何标注数据即可识别针对不同语言输入所激活的神经元。利用$\texttt{PLND}$，我们通过大量实验验证了$\texttt{MWork}$，这些实验涉及在不同层和结构中停用语言特定神经元。此外，$\texttt{MWork}$允许使用小型数据集对语言特定神经元进行微调，从而增强特定语言的多语言能力而不影响其他语言。该方法仅使用$400$个文档，就在所有任务中使高资源语言的平均性能提升了$3.6\%$，低资源语言提升了$2.3\%$。