If large language models operate in a universal semantic space, then switching between languages should require only a simple activation offset. To test this, we take multilingual in-context learning as a case study, where few-shot demonstrations are provided in English but the test query is in a target language. We propose language vectors, computed as the mean activation difference between parallel source and target language examples at a particular layer, and added as an offset to hidden states at inference time to shift the model's internal representations toward the target language. We evaluate our method across three multilingual tasks spanning 19 languages and three models. Our results show consistent improvements on multilingual in-context learning over baselines across all tasks and languages tested, demonstrating that a simple activation offset is sufficient to redirect a model's language mode without any parameter updates. Beyond performance, the vectors encode interpretable linguistic structure, with closely related languages forming tight clusters and vectors transferring across tasks, suggesting that language identity occupies separable and structured directions in a model's activation space.

翻译：若大语言模型运行于通用语义空间中，语言切换应仅需简单的激活偏移。为验证该假设，我们以多语言上下文学习为例——其中少样本示例以英文呈现，而测试查询使用目标语言——提出语言向量法：计算特定层并行源语言与目标语言示例的平均激活差异，在推理时作为偏移量添加至隐藏状态，以将模型内部表征转向目标语言。我们在覆盖19种语言的三项多语言任务及三个模型上评估该方法。结果表明，相较基线方法，该方法在所有测试任务与语言的多语言上下文学习性能上呈现一致性提升，验证了无需参数更新的简单激活偏移即可重定向模型语言模式。除性能优势外，该向量编码了可解释的语言结构——相近语言形成紧密聚类，且向量可跨任务迁移，表明语言身份在模型激活空间中占据可分离的结构化方向。