Script Event Prediction (SEP) aims to predict the subsequent event for a given event chain from a candidate list. Prior research has achieved great success by integrating external knowledge to enhance the semantics, but it is laborious to acquisite the appropriate knowledge resources and retrieve the script-related knowledge. In this paper, we regard public pre-trained language models as knowledge bases and automatically mine the script-related knowledge via prompt-learning. Still, the scenario-diversity and label-ambiguity in scripts make it uncertain to construct the most functional prompt and label token in prompt learning, i.e., prompt-uncertainty and verbalizer-uncertainty. Considering the innate ability of Gaussian distribution to express uncertainty, we deploy the prompt tokens and label tokens as random variables following Gaussian distributions, where a prompt estimator and a verbalizer estimator are proposed to estimate their probabilistic representations instead of deterministic representations. We take the lead to explore prompt-learning in SEP and provide a fresh perspective to enrich the script semantics. Our method is evaluated on the most widely used benchmark and a newly proposed large-scale one. Experiments show that our method, which benefits from knowledge evoked from pre-trained language models, outperforms prior baselines by 1.46\% and 1.05\% on two benchmarks, respectively.
翻译:脚本事件预测(Script Event Prediction, SEP)旨在从候选列表中预测给定事件链的后续事件。先前研究通过整合外部知识增强语义取得了巨大成功,但获取合适的知识资源并检索脚本相关知识既费时又费力。本文将公开的预训练语言模型视为知识库,并借助提示学习自动挖掘脚本相关知识。然而,脚本中的场景多样性和标签模糊性使得在提示学习中构建最有效的提示词和标签词具有不确定性,即提示不确定性和言语器不确定性。考虑到高斯分布在表达不确定性方面的固有优势,我们将提示词和标签词部署为遵循高斯分布的随机变量,并提出提示估计器和言语器估计器来估计其概率表示而非确定性表示。我们率先探索了提示学习在SEP中的应用,并为丰富脚本语义提供了全新视角。该方法在广泛使用的基准数据集和新提出的大规模数据集上进行了评估。实验表明,得益于预训练语言模型所激发的知识,我们的方法在两个基准数据集上分别以1.46%和1.05%的提升优于先前基线方法。