We introduce a new zero resource code-switched speech benchmark designed to directly assess the code-switching capabilities of self-supervised speech encoders. We showcase a baseline system of language modeling on discrete units to demonstrate how the code-switching abilities of speech encoders can be assessed in a zero-resource manner. Our experiments encompass a variety of well-known speech encoders, including Wav2vec 2.0, HuBERT, XLSR, etc. We examine the impact of pre-training languages and model size on benchmark performance. Notably, though our results demonstrate that speech encoders with multilingual pre-training, exemplified by XLSR, outperform monolingual variants (Wav2vec 2.0, HuBERT) in code-switching scenarios, there is still substantial room for improvement in their code-switching linguistic abilities.

翻译：我们提出了一种新的零资源语码转换语音基准，旨在直接评估自监督语音编码器的语码转换能力。我们展示了一个基于离散单元语言建模的基线系统，以说明如何在零资源条件下评估语音编码器的语码转换能力。实验涵盖了多种知名语音编码器，包括Wav2vec 2.0、HuBERT及XLSR等。我们研究了预训练语言种类和模型规模对基准性能的影响。值得注意的是，尽管我们的结果表明，以XLSR为代表的多语言预训练语音编码器在语码转换场景中优于单语言变体（Wav2vec 2.0、HuBERT），但其语码转换语言能力仍有显著的提升空间。