In the realm of large language models (LLMs), enhancing instruction-following capability often involves curating expansive training data. This is achieved through two primary schemes: i) Scaling-Inputs: Amplifying (input, output) pairs per task instruction, aiming for better instruction adherence. ii) Scaling Input-Free Tasks: Enlarging tasks, each composed of an (instruction, output) pair (without requiring a separate input anymore). However, LLMs under Scaling-Inputs tend to be overly sensitive to inputs, leading to misinterpretation or non-compliance with instructions. Conversely, Scaling Input-Free Tasks demands a substantial number of tasks but is less effective in instruction following when dealing with instances in Scaling-Inputs. This work introduces MUFFIN, a new scheme of instruction-following dataset curation. Specifically, we automatically Scale Tasks per Input by diversifying these tasks with various input facets. Experimental results across four zero-shot benchmarks, spanning both Scaling-Inputs and Scaling Input-Free Tasks schemes, reveal that LLMs, at various scales, trained on MUFFIN generally demonstrate superior instruction-following capabilities compared to those trained on the two aforementioned schemes.

翻译：在大语言模型（LLMs）领域，增强指令遵循能力通常涉及整理大规模训练数据，主要通过两种方案实现：i）输入扩展：对每个任务指令增加（输入，输出）对，旨在提升指令遵循效果；ii）无输入任务扩展：增加任务数量，每个任务仅包含（指令，输出）对（不再需要单独的输入）。然而，采用输入扩展方案的LLMs容易对输入过度敏感，导致指令误解或不执行。相反，无输入任务扩展方案虽需要大量任务，但在处理输入扩展中的实例时，指令遵循效果较差。本文提出MUFFIN——一种新的指令遵循数据集整理方案。具体而言，我们通过使用多样化的输入面来自动扩展每个输入下的任务。在四个零样本基准测试（涵盖输入扩展与无输入任务扩展两种方案）上的实验结果表明，不同规模的LLMs在使用MUFFIN训练后，其指令遵循能力通常优于上述两种方案。