Self-rationalizing models that also generate a free-text explanation for their predicted labels are an important tool to build trustworthy AI applications. Since generating explanations for annotated labels is a laborious and costly pro cess, recent models rely on large pretrained language models (PLMs) as their backbone and few-shot learning. In this work we explore a self-training approach leveraging both labeled and unlabeled data to further improve few-shot models, under the assumption that neither human written rationales nor annotated task labels are available at scale. We introduce a novel dual-teacher learning framework, which learns two specialized teacher models for task prediction and rationalization using self-training and distills their knowledge into a multi-tasking student model that can jointly generate the task label and rationale. Furthermore, we formulate a new loss function, Masked Label Regularization (MLR) which promotes explanations to be strongly conditioned on predicted labels. Evaluation on three public datasets demonstrate that the proposed methods are effective in modeling task labels and generating faithful rationales.

翻译：自我合理化模型能够为其预测标签生成自由文本解释，是构建可信赖人工智能应用的重要工具。由于为标注标签生成解释是一个费时费力的过程，现有模型通常依赖大型预训练语言模型作为骨干网络并采用少样本学习。本研究探索了一种利用标注数据与未标注数据的自训练方法，在假设人工撰写的理由和标注任务标签均无法大规模获取的条件下，进一步改进少样本模型。我们提出了一种新颖的双教师学习框架，通过自训练学习任务预测与理由生成两个专门化教师模型，并将其知识蒸馏至一个能够联合生成任务标签与理由的多任务学生模型中。此外，我们设计了一种新的损失函数——掩码标签正则化，该函数促使模型生成的解释与预测标签之间存在强依赖关系。在三个公开数据集上的评估结果表明，所提方法能有效建模任务标签并生成忠实的理由。