This paper introduces WARDEN, an early language model system capable of transcribing and translating Wardaman, an endangered Australian indigenous language into English. The significant challenge we face is the lack of large-scale training data: in fact, we only have 6 hours of annotated audio. Therefore, while it is common practice to train a single model for transcription and translation using large datasets (like English to French), this practice is no longer viable in the Wardaman to English context. To tackle the low-resource challenge, we design WARDEN to have separate transcription and translation models: WARDEN first turns a Wardaman audio input into phonemic transcription, and then the transcription into English translation. Further, we propose two useful techniques to enhance performance. For transcription, we initialize the Wardaman token from Sundanese, a language that shares similar phonemes with Wardaman, to accelerate fine-tuning of the transcription model. For translation, we compile a Wardaman-English dictionary from expert annotations, and provide this domain-specific knowledge to a large language model (LLM) to reason and decide the final output. We empirically demonstrate that this two-stage design works better than data-hungry unified approaches in extremely low data settings. Using a mere 6 hours of annotated data, WARDEN outperforms larger open-source and proprietary models and establishes a strong baseline. Data and code are available.

翻译：本文介绍WARDEN系统，这是一个能够将濒危澳大利亚土著语言Wardaman转录并翻译为英语的早期语言模型。我们面临的核心挑战是缺乏大规模训练数据：实际上仅有6小时带标注的音频数据。因此，虽然使用大规模数据集（如英语到法语）训练单一模型进行转录与翻译是常见做法，但在Wardaman到英语场景中已不再适用。为解决低资源挑战，我们设计WARDEN采用分离的转录与翻译模型：系统首先将Wardaman音频输入转换为音素转录，再将转录结果转为英文翻译。此外，我们提出两种增强性能的有效技术。在转录方面，从与Wardaman共享相似音素的巽他语初始化Wardaman词元，以加速转录模型的微调。在翻译方面，基于专家标注构建Wardaman-英语词典，并将该领域知识提供给大语言模型(LLM)进行推理并生成最终输出。实验证明，在数据极度匮乏条件下，这种两阶段设计优于需要大量数据的统一方法。仅凭6小时标注数据，WARDEN的性能即超越更大规模的开源与专有模型，并建立了强基准。数据和代码已公开。