Small language models (SLMs) are efficient and scalable, but their multilingual capabilities degrade severely at sub-billion scales, especially for Southeast Asian (SEA) languages. We introduce DuDi, a dual-signal multilingual distillation framework that combines an online sequence-level signal with off-policy and on-policy token-level signals. DuDi further uses a cross-lingual verbalizer to refine teacher feedback and improve teacher-student transferability in multilingual settings. Experiments on SEA-HELM across multiple model families, scales, and teacher-student settings show that DuDi consistently outperforms competitive distillation baselines. Ablations and analyses confirm that sequence-level optimization, token-level supervision, and cross-lingual verbalization provide complementary and transferable learning signals for multilingual SLMs.

翻译：小型语言模型（SLMs）虽高效可扩展，但在亚十亿参数规模下的多语言能力严重退化，尤其对东南亚（SEA）语言表现更为突出。本文提出DuDi——一种双信号多语言知识蒸馏框架，该框架融合了在线序列级信号与离策略及在策略两种词元级信号。DuDi进一步采用跨语言动词化器优化教师反馈，提升多语言环境下师生模型的知识迁移效能。在SEA-HELM基准上，针对多种模型族、参数量级及师生配置的实验表明，DuDi始终优于现有蒸馏基线。消融实验与深入分析证实，序列级优化、词元级监督与跨语言动词化机制可为多语言SLMs提供互补且可迁移的学习信号。