We present ESPnet-SpeechLM, an open toolkit designed to democratize the development of speech language models (SpeechLMs) and voice-driven agentic applications. The toolkit standardizes speech processing tasks by framing them as universal sequential modeling problems, encompassing a cohesive workflow of data preprocessing, pre-training, inference, and task evaluation. With ESPnet-SpeechLM, users can easily define task templates and configure key settings, enabling seamless and streamlined SpeechLM development. The toolkit ensures flexibility, efficiency, and scalability by offering highly configurable modules for every stage of the workflow. To illustrate its capabilities, we provide multiple use cases demonstrating how competitive SpeechLMs can be constructed with ESPnet-SpeechLM, including a 1.7B-parameter model pre-trained on both text and speech tasks, across diverse benchmarks. The toolkit and its recipes are fully transparent and reproducible at: https://github.com/espnet/espnet/tree/speechlm.

翻译：我们推出ESPnet-SpeechLM，这是一个旨在普及语音语言模型（SpeechLM）及语音驱动智能体应用开发的开源工具包。该工具包通过将语音处理任务统一框架化为通用序列建模问题，实现了任务标准化，涵盖数据预处理、预训练、推理和任务评估的完整工作流程。借助ESPnet-SpeechLM，用户可以轻松定义任务模板并配置关键参数，实现无缝、高效的SpeechLM开发。工具包通过为工作流各阶段提供高度可配置的模块，确保了灵活性、高效性和可扩展性。为展示其能力，我们提供了多个用例，演示如何利用ESPnet-SpeechLM构建具有竞争力的语音语言模型，包括一个在文本和语音任务上预训练、参数量达17亿的模型，并在多样化基准测试中进行了验证。该工具包及其完整实现方案完全透明且可复现，详见：https://github.com/espnet/espnet/tree/speechlm。