Training Large Language Models (LLMs) with high multilingual coverage is becoming increasingly important -- especially when monolingual resources are scarce. Recent studies have found that LLMs process multilingual inputs in shared concept spaces, thought to support generalization and cross-lingual transfer. However, these prior studies often do not use causal methods, lack deeper error analysis or focus on the final model only, leaving open how these spaces emerge during training. We investigate the development of language-agnostic concept spaces during pretraining of EuroLLM through the causal interpretability method of activation patching. We isolate cross-lingual concept representations, then inject them into a translation prompt to investigate how consistently translations can be altered, independently of the language. We find that shared concept spaces emerge early} and continue to refine, but that alignment with them is language-dependent}. Furthermore, in contrast to prior work, our fine-grained manual analysis reveals that some apparent gains in translation quality reflect shifts in behavior -- like selecting senses for polysemous words or translating instead of copying cross-lingual homographs -- rather than improved translation ability. Our findings offer new insight into the training dynamics of cross-lingual alignment and the conditions under which causal interpretability methods offer meaningful insights in multilingual contexts.

翻译：训练具有高多语言覆盖率的大型语言模型正变得日益重要——尤其是在单语资源稀缺的情况下。近期研究发现，LLMs在多语言输入处理中运用共享概念空间，这被认为有助于泛化与跨语言迁移。然而，这些先前研究往往未采用因果方法、缺乏深入的错误分析或仅关注最终模型，使得这些空间在训练过程中如何形成的问题尚未解决。我们通过激活修补这一因果可解释性方法，研究了EuroLLM预训练过程中语言无关概念空间的发展。我们分离出跨语言概念表征，随后将其注入翻译提示中，以探究翻译结果能在多大程度上被独立于语言地一致改变。我们发现共享概念空间早期即已出现并持续优化，但其对齐程度具有语言依赖性。此外，与先前研究不同，我们的细粒度人工分析表明，某些翻译质量的表面提升实际反映了行为模式的转变——例如为多义词选择特定义项，或对跨语言同形词进行翻译而非直接复制——而非翻译能力的真实提升。我们的发现为跨语言对齐的训练动态，以及因果可解释性方法在多语言语境中提供有效洞察的条件，提供了新的理解。