With the burgeoning development in the realm of large language models (LLMs), the demand for efficient incremental training tailored to specific industries and domains continues to increase. Currently, the predominantly employed frameworks lack modular design, it often takes a lot of coding work to kickstart the training of LLM. To address this, we present "LMTuner", a highly usable, integrable, and scalable system for training LLMs expeditiously and with minimal user-input. LMTuner comprises three main modules - the Interaction, Training, and Inference Modules. We advocate that LMTuner's usability and integrality alleviate the complexities in training large language models. Remarkably, even a novice user could commence training large language models within five minutes. Furthermore, it integrates DeepSpeed frameworks and supports Efficient Fine-Tuning methodologies like Low Rank Adaptation (LoRA), Quantized LoRA (QLoRA), etc., enabling the training of language models scaling from 300M to a whopping 130B parameters using a single server. The LMTuner's homepage (https://wengsyx.github.io/LMTuner/)and screencast video (https://youtu.be/nsXmWOmN3rE) are now publicly available.

翻译：随着大型语言模型领域的蓬勃发展，针对特定行业和领域的高效增量训练需求持续增长。当前主流框架缺乏模块化设计，启动大型语言模型训练往往需要大量编码工作。为此，我们提出LMTuner——一个高可用性、可集成且可扩展的系统，能以最小用户输入快速训练大型语言模型。LMTuner包含三大核心模块：交互模块、训练模块和推理模块。我们认为LMTuner的易用性与完整性降低了大型语言模型训练的复杂性。值得注意的是，即使是新手用户也能在五分钟内完成大型语言模型的训练启动。该系统集成了DeepSpeed框架，并支持低秩自适应（LoRA）、量化LoRA（QLoRA）等高效微调方法，可实现在单台服务器上训练参数规模从3亿到1300亿的语言模型。LMTuner主页（https://wengsyx.github.io/LMTuner/）及操作演示视频（https://youtu.be/nsXmWOmN3rE）现已公开。