Instruction tuning has emerged as a crucial process for harnessing the capabilities of large language models (LLMs) by providing explicit task instructions, leading to improved performance in various tasks. However, prevalent text-to-text instruction tuning (TextTuning) methods suffer from limitations in generalization, robustness, and controllability due to the ambiguity and lack of explicit structure in tasks. In this paper, we propose JsonTuning, a novel structure-to-structure approach for instruction tuning. By leveraging the versatility and structured nature of JSON to represent tasks, JsonTuning enhances generalization by helping the model understand essential task elements and their relations, improves robustness by minimizing ambiguity, and increases controllability by providing explicit control over the output. We conduct a comprehensive comparative study with diverse language models and evaluation benchmarks. Experimental results show that JsonTuning outperforms TextTuning in various applications, showcasing improved performance, adaptability, robustness, and controllability. By overcoming the limitations of TextTuning, JsonTuning demonstrates significant potential for more effective and reliable LLMs capable of handling diverse scenarios.

翻译：指令微调已成为通过提供明确任务指令来发挥大型语言模型（LLM）能力的关键过程，从而提升各项任务的性能。然而，由于任务中存在的模糊性和缺乏显式结构，主流的文本到文本指令微调（TextTuning）方法在泛化性、鲁棒性和可控性方面存在局限。本文提出一种新颖的结构到结构指令微调方法——JsonTuning。通过利用JSON的通用性和结构化特性来表示任务，JsonTuning帮助模型理解任务关键要素及其关系，从而增强泛化性；通过减少模糊性提升鲁棒性；并通过提供对输出的显式控制增强可控性。我们基于多种语言模型和评估基准开展了全面的对比研究。实验结果表明，JsonTuning在各类应用中均优于TextTuning，展现出更优的性能、适应性、鲁棒性和可控性。通过克服TextTuning的局限性，JsonTuning在开发能够处理多样化场景的更有效、更可靠的LLM方面展现出巨大潜力。