Traditional supervised learning mostly works on individual tasks and requires training on a large set of task-specific examples. This paradigm seriously hinders the development of task generalization since preparing a task-specific example set is costly. To build a system that can quickly and easily generalize to new tasks, task instructions have been adopted as an emerging trend of supervision recently. These instructions give the model the definition of the task and allow the model to output the appropriate answer based on the instructions and inputs. However, task instructions are often expressed in different forms, which can be interpreted from two threads: first, some instructions are short sentences and are pretrained language model (PLM) oriented, such as prompts, while other instructions are paragraphs and are human-oriented, such as those in Amazon MTurk; second, different end-users very likely explain the same task with instructions of different textual expressions. A robust system for task generalization should be able to handle any new tasks regardless of the variability of instructions. However, the system robustness in dealing with instruction-driven task generalization is still unexplored. This work investigates the system robustness when the instructions of new tasks are (i) manipulated, (ii) paraphrased, or (iii) from different levels of conciseness. To our knowledge, this is the first work that systematically studies how robust a PLM is when it is supervised by instructions with different factors of variability.

翻译：传统监督学习主要针对单一任务，需要依赖大量任务特定示例进行训练。这种范式严重阻碍了任务泛化能力的发展，因为构建任务特定示例集的成本高昂。为构建能快速泛化至新任务的系统，近年来任务指令已成为一种新兴的监督范式。这类指令向模型提供任务定义，要求模型根据指令和输入输出相应答案。然而，任务指令常以不同形式呈现，可从两个维度解读：其一，部分指令是面向预训练语言模型（PLM）的短句（如提示词），而其他指令则是面向人类的段落式描述（如Amazon MTurk中的指令）；其二，不同终端用户很可能会使用不同文本表述的指令来解释同一任务。一个鲁棒的任务泛化系统应能应对任意新任务，不受指令多样性影响。但目前系统在处理指令驱动任务泛化时的鲁棒性问题尚未被探索。本研究系统分析了当新任务指令经历（i）篡改、（ii）复述改写或（iii）不同简洁度表达时的系统鲁棒性。据我们所知，这是首个系统研究PLM在受包含不同变异因素的指令监督时鲁棒性的工作。