We present symbol tuning - finetuning language models on in-context input-label pairs where natural language labels (e.g., "positive/negative sentiment") are replaced with arbitrary symbols (e.g., "foo/bar"). Symbol tuning leverages the intuition that when a model cannot use instructions or natural language labels to figure out a task, it must instead do so by learning the input-label mappings. We experiment with symbol tuning across Flan-PaLM models up to 540B parameters and observe benefits across various settings. First, symbol tuning boosts performance on unseen in-context learning tasks and is much more robust to underspecified prompts, such as those without instructions or without natural language labels. Second, symbol-tuned models are much stronger at algorithmic reasoning tasks, with up to 18.2% better performance on the List Functions benchmark and up to 15.3% better performance on the Simple Turing Concepts benchmark. Finally, symbol-tuned models show large improvements in following flipped-labels presented in-context, meaning that they are more capable of using in-context information to override prior semantic knowledge.

翻译：我们提出符号调优——一种针对语言模型的微调方法，通过在其上下文输入-标签对中将自然语言标签（如“正面/负面情感”）替换为任意符号（如“foo/bar”）来实现。符号调优利用了以下直觉：当模型无法借助指令或自然语言标签理解任务时，它必须通过学习输入-标签映射来完成。我们使用参数规模高达540B的Flan-PaLM模型进行符号调优实验，并在多种设置中观察到其优势。首先，符号调优提升了模型在未见过的上下文学习任务上的性能，且对不明确的提示（例如缺少指令或自然语言标签的提示）具有更强的鲁棒性。其次，经过符号调优的模型在算法推理任务上表现显著提升，在列表函数基准测试中性能提升达18.2%，在简单图灵概念基准测试中性能提升达15.3%。最后，符号调优的模型在遵循上下文翻转标签方面表现出大幅改进，这意味着它们能更有效地利用上下文信息覆盖先验语义知识。