We present symbol tuning - finetuning language models on in-context input-label pairs where natural language labels (e.g., "positive/negative sentiment") are replaced with arbitrary symbols (e.g., "foo/bar"). Symbol tuning leverages the intuition that when a model cannot use instructions or natural language labels to figure out a task, it must instead do so by learning the input-label mappings. We experiment with symbol tuning across Flan-PaLM models up to 540B parameters and observe benefits across various settings. First, symbol tuning boosts performance on unseen in-context learning tasks and is much more robust to underspecified prompts, such as those without instructions or without natural language labels. Second, symbol-tuned models are much stronger at algorithmic reasoning tasks, with up to 18.2% better performance on the List Functions benchmark and up to 15.3% better performance on the Simple Turing Concepts benchmark. Finally, symbol-tuned models show large improvements in following flipped-labels presented in-context, meaning that they are more capable of using in-context information to override prior semantic knowledge.

翻译：我们提出符号调优——对语言模型进行基于上下文输入-标签对的微调，其中自然语言标签（如“正面/负面情感”）被替换为任意符号（如“foo/bar”）。符号调优利用了一个直觉：当模型无法通过指令或自然语言标签识别任务时，它必须通过学习输入-标签映射来完成任务。我们对参数规模高达540B的Flan-PaLM模型进行了符号调优实验，并在多种设置下观察到性能提升。首先，符号调优增强了模型在未见过的上下文学习任务上的表现，并且对指令缺失或自然语言标签缺失等不明确提示具有更强的鲁棒性。其次，符号调优后的模型在算法推理任务上表现显著提升，在列表函数基准测试中性能提升最高达18.2%，在简单图灵概念基准测试中性能提升最高达15.3%。最后，符号调优模型在遵循上下文提供的翻转标签方面表现出大幅改进，意味着它们能更有效地利用上下文信息来覆盖先验语义知识。