Modern LLMs typically require multistage training pipelines to achieve strong downstream performance, with post-training serving as the main interface for adapting open-weight models. We introduce torchtune, a PyTorch-native library designed to streamline the post-training lifecycle of LLMs, enabling efficient fine-tuning, experimentation, and deployment-oriented workflows. Unlike many existing fine-tuning frameworks, which often optimize for ease of use, specialized recipes, or hardware efficiency at the cost of transparency and extensibility, torchtune emphasizes modularity, hackability, and direct access to the underlying PyTorch components. In this paper, we present the design principles behind torchtune, describe how they are reflected in its model builders, training recipes, and distributed training stack, and evaluate the library across representative post-training settings. We compare against popular fine-tuning frameworks, including Axolotl and Unsloth, and show that torchtune provides strong performance and memory efficiency across many settings while remaining flexible enough for rapid research iteration. These results position torchtune as a practical foundation for reproducible LLMs post-training research.

翻译：现代大型语言模型通常需要多阶段训练流程才能达到强大的下游性能，而后训练是适配开放权重模型的主要接口。我们提出torchtune，一个原生PyTorch库，旨在简化LLM的后训练生命周期，支持高效微调、实验和面向部署的工作流程。与许多现有微调框架（这些框架常以易用性、特殊方法或硬件效率为优化目标，但牺牲了透明性和可扩展性）不同，torchtune强调模块化、可定制性以及对底层PyTorch组件的直接访问。本文阐述了torchtune的设计原则，描述了这些原则如何体现在其模型构建器、训练方法和分布式训练栈中，并在代表性后训练场景下评估该库。我们对比了包括Axolotl和Unsloth在内的主流微调框架，结果表明torchtune在多种场景下兼具强劲性能与内存效率，同时保持足够灵活性以支持快速研究迭代。这些成果使torchtune成为可复现LLM后训练研究的实用基础。