Instruction tuning has emerged as a crucial process for harnessing the capabilities of large language models (LLMs) by providing explicit task instructions, leading to improved performance in various tasks. However, prevalent text-to-text instruction tuning (TextTuning) methods suffer from limitations in generalization, robustness, and controllability due to the ambiguity and lack of explicit structure in tasks. In this paper, we propose JsonTuning, a novel structure-to-structure approach for instruction tuning. By leveraging the versatility and structured nature of JSON to represent tasks, JsonTuning enhances generalization by helping the model understand essential task elements and their relations, improves robustness by minimizing ambiguity, and increases controllability by providing explicit control over the output. We conduct a comprehensive comparative study with diverse language models and evaluation benchmarks. Experimental results show that JsonTuning outperforms TextTuning in various applications, showcasing improved performance, adaptability, robustness, and controllability. By overcoming the limitations of TextTuning, JsonTuning demonstrates significant potential for more effective and reliable LLMs capable of handling diverse scenarios.

翻译：指令微调已成为通过提供明确任务指令来释放大型语言模型（LLM）能力的关键过程，从而提升其在各类任务中的性能表现。然而，当前主流的文本到文本指令微调（TextTuning）方法由于任务表述的模糊性和缺乏显式结构，在通用性、鲁棒性和可控性方面存在局限性。本文提出JsonTuning，一种新颖的结构到结构指令微调方法。通过利用JSON的通用性和结构化特性来表示任务，JsonTuning能够帮助模型理解任务核心要素及其关联关系以增强通用性，通过最小化歧义性提升鲁棒性，并通过提供对输出的显式控制增强可控性。我们基于多种语言模型和评估基准进行了全面对比研究。实验结果表明，JsonTuning在各类应用中均优于TextTuning，展现出更优异的性能表现、适应能力、鲁棒性和可控性。通过克服TextTuning的局限性，JsonTuning为构建更高效、更可靠的能处理多样化场景的LLM展现了巨大潜力。